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PRELIMINARY AMENDMENT 

15 Sir: 

Please amend the above-identified patent application as follows: 
In the Specification: 

SPECIFICATION 

TITLE OF THE INVENTION 

20 IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND 
RECORDING MEDIUM 
BACKGROUND OF THE INVENTION 
The present invention relates to an image processing apparatus, an image 
processing method, and a recording medium which are suitable for s>Tithesizing a hair 
25 image expressed by computer graphics with a three-dimensional shape image of a head 
part. 

Conventionally, as a method for expressing human hair by computer graphics 
(hereinafter abbreviated as CG), it is useful to take a method in which an entire hair 
style is approximated to a polygon model, as shown in FIG. lA, and a texture of hairs 
30 is adhered to the model, or a method in which rendering is performed with use of an 



anisotropic reflection model which has different reflection intensities respectively with 
respect to light source positions or viewpoint positions. 

These methods are described in "Hair expression using an anisotropic 
reflection model" by Yamana and Suenaga, Singakugihou, PRU87-3, pp. 115 to 120, 
5 1989. In these methods, expression of a hair style is just the shape of a polygon model 
itself, and the hair style can be changed by changing the shape of the polygon model. 

A proposal has been made for a method of expressing hair by a particle system. 
In the particle system, when a large quantity of particle objects are dealt with, 
characteristics such as weight and the like of each particle are given as parameters, and 
10 particles are generated under a certain rule. Thus, a large quantity of particle objects 
can be expressed easily. 

Meanwhile, the present applicant has proposed a method of approximating 
every hair of a hairstyle to a curve, as shown in FIG. IB, in Japanese Patent 
Application No. 10-164545, for example. This method is successful in natural 
1 5 expression of a hair style image. 

A method of designing a hair style by editing together a plurality of hairs 
approximated to a cxirve is described in "Development of a hair design system using 
tuft models" by Kishi, Saegusa, and Morishima, in Shingakugihou, MVE97-102, pp. 
67-74, 1998. 

20 It is difficult for the method of approximating the entire hair style to a polygon 

model to express a compKcated hair style. In this method, movements (flows) of hair 
cannot be expressed. 

On the other side, the method of expressing hair by a particle system and the 
method of approximating every hair to a geometric figure can express any arbitrary 

25 hair style and movements of hair. However, in the method of expressing hair by the 
particle system, parameters for particles expressing hairs must be controlled to control 
the hair style, so that intuitive control is difficult. It is therefore difficult to generate an 
intended hair style. 

In the method described in "Development of a hair design system using tuft 
30 models" by Kishi, Saegusa, and Morishima, in Shingakugihou, MVE97-102, pp. 67- 
74, 1998, an intended hair style can be generated relatively easily. This method, 
however, has a problem in that hair cannot be changed to intended hair styles 
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corresponding to a plurality of different head shapes although hair corresponding only 
to one fixed head shape can be changed to an intended hair style. 

SUMMARY OF THE INVENTION 
One of the objects of the present invention is changing a hair style consisting of 
5 hairs of which is expressed by CG, so as to correspond to a number of different head 
shapes. 

The present invention provides an image processing apparatus for synthesizing 
a hair image with a three-dimensional shape image of a head part, to generate a hair- 
style-matched image, including: a storage part for storing a hair-style data piece; a read 

10 out part for reading out the hair-style data piece stored in the storage part; a mapping 
part for mapping a hair data piece corresponding to a hair and contained in the hair- 
style data piece read out by the read out part, at a predetermined position on the three- 
dimensional shape image of the head part; and a generation part for generating the 
hair-style-matched image, based on a mapping result obtained by the mapping part. 

1 5 As the hair data piece, it is possible to use three-dimensional curve data which 

is constructed by a number of control points. 

As the hair data piece, it is possible to use polygon data which is constructed 
by a number of polygons. 

The storage part stores the hair-style data piece constructed by hair data pieces 

20 arranged on a two-dimensional array. In particular, the storage part stores the hair- 
style data piece constructed by data pieces arranged on a two-dimensional array 
corresponding to a projected image obtained by projecting a three-dimensional hair 
style expressed by a columnar coordinate system, on a two-dimensional coordinate 
system. 

25 The apparatus according to the present invention further includes an 

interpolation part for interpolating the hair data piece at the predetermined position, 
with use of a hair data piece close to the predetermined position. The interpolation 
part interpolates the hair data piece, based on group information contained in the hair 
data piece close to the predetermined position. 

30 The mapping part includes a coordinate conversion part for converting the hair 

data piece expressed by a first coordinate system into data of a second coordinate 
system by which the three-dimensional shape image of the head part is expressed. 
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The coordinate conversion part converts tiie hair data piece expressed by the 
first coordinate system into data of a local coordinate system having an origin on a 
surface of the head part, and thereafter converts the data into data of the second 
coordinate system by which the three-dimensional shape image of the head part is 
expressed. 

When the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 
image of the head part is expressed, the coordinate conversion part executes at least 
one of coordinate axis rotation and origin shift, based on a random number. 

Also, the present invention provides an image processing method for 
synthesizing a hair image with a three-dimensional shape image of a head part, to 
generate a hair-style-matched image, including: a storage step of storing a hair-style 
data piece; a read out step of reading out the hair-style data piece stored by processing 
in the storage step; a mapping step of mapping a hair data piece corresponding to a 
hair and contained in the hair-style data piece read out by processing in the read out 
step, at a predetermined position on the three-dimensional shape image of the head 
part; and a generation step of generating the hair-style-matched image, based on a 
mapping result obtained by processing in the mapping step. 

In the processing in the storage step according to the present invention, the 
hair-style data piece constructed by hair data pieces arranged on a two-dimensional 
array is stored. 

Also, in the processing in the storage step, the hair-style data piece constructed 
by data pieces arranged on a two-dimensional array corresponding to a projected 
image obtained by projecting a three-dimensional hair style expressed by a columnar 
coordinate system, on a two-dimensional coordinate system, is stored. 

The method according to the present invention further includes an interpolation 
step of interpolating the hair data piece at the predetermined position, with use of a 
hair data piece close to the predetermined position. 

In processing in the interpolation step, the hair data piece is interpolated, based 
on group information contained in the hair data piece close to the predetermined 
position. 



The processing in the mapping step includes a coordiaate conversion step of 
converting the hair data piece expressed by a first coordinate system into data of a 
second coordinate system by which the three-dimensional shape image of the head part 
is expressed. 

5 In processing in the coordinate conversion step, the hair data piece expressed 

by the first coordinate system is converted into data of a local coordinate system 
having an origin on a surface of the head part, and thereafter, the data is converted into 
data of the second coordinate system by which the three-dimensional shape image of 
the head part is expressed. 
1 0 When the hair data piece expressed by the first coordinate system is converted 

into the data of the second coordinate system by which the three-dimensional shape 
image of the head part is expressed, at least one of coordinate axis rotation and origin 
shift is executed, based on a random mmiber, in processing in the coordinate 
conversion step. 

15 Further, the present invention provides a recording medium which records a 

computer-readable program for image processing of synthesizing a hair image with a 
three-dimensional shape image of a head part, to generate a hair-style-matched image, 
the program including: a storage step of storing a hair-style data piece; a read out step 
of reading out the hair-style data piece stored by processing in the storage step; a 

20 mapping step of mapping a hair data piece corresponding to a hair and contained in the 
hair-style data piece read out by processing in the read out step, at a predetermined 
position on the three-dimensional shape image of the head part; and a generation step 
of generating the hair-style-matched image, based on a mapping result obtained by 
processing in the mapping step. 

25 In the processing in the storage step, the hair-style data piece constructed by 

the hair data pieces arranged on a two-dimensional array is stored. 

In the processing in the storage step, the hair-style data piece constructed by 
data pieces arranged on a two-dimensional array corresponding to a projected image 
obtained by projecting a three-dunensional hair style expressed by a columnar 

30 coordinate system, on a two-dimensional coordinate system, is stored. 
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According to the present invention, the program fiirther includes an 
interpolation step of interpolating the hair data piece at the predetermined position, 
with use of a hair data piece close to the predetermined position. 

In processing in the interpolation step, the hair data piece is interpolated, based 
5 on group information contained in the hair data piece close to the predetermined 
position. 

The processing in the mapping step includes a coordinate conversion step of 
converting the hair data piece expressed by a first coordinate system into data of a 
second coordinate system by which the three-dimensional shape image of the head part 
10 is expressed. 

In processing in the coordinate conversion step, the hair data piece expressed 
by the first coordinate system is converted into data of a local coordinate system 
having aa origin on a surface of the head part, and thereafter, the data is converted into 
data of the second coordinate system by which the three-dimensional shape image of 
1 5 the head part is expressed. 

When the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 
image of the head part is expressed, at least one of coordinate axis rotation and origin 
shift is executed, based on a random nimiber, in processing in the coordinate 
20 conversion step. 

In the above-described image processing apparatus and method according to 
the present invention, hair-style data is stored, and the stored hair-style data is read out. 
Hair data corresponding to a hair and included in the read out hair-style data is mapped 
at a predetermined position on a three-dimensional shape image of a head part. Based 
25 on the mapping results, a hair-style-matched image is generated. The stored hair-style 
data is constructed by pieces of hair data arranged on a two-dimensional array. 

Additional features and advantages of the present invention are described in, 
and will be apparent firom, the following Detailed Description of the Livention and the 
Figures. 

30 BRIEF DESCRIPTION OF THE FIGURES 

FIGS. lA and IB are views showing an example of conventional hair 
expression. 
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FIG. 2 is a block diagram showing a structural example of a hair synthesis 
system to which the present invention is apphed. 

FIG. 3 is a flowchart explaining operation of the hair synthesis system. 

FIG. 4 is a view for explaining the operation of the hair synthesis system. 
5 FIG. 5 is a block diagram showing a structural example of processing in the 

hair-style mapping processing section shown in FIG. 2. 

FIG. 6 is a flowchart for explaining processing in the hair-style mapping 
processing section. 

FIGS. 7 A and 7B are views for explaining hair-style data in which hair data is 
10 two-dimensionally arrayed. 

FIG. 8 is a view for explaining mapping position calculation in the step S13 
shown in FIG. 6. 
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U FIG. 9 is a view for explaining hair data interpolation processing in the step 



51 5 shown in FIG. 6. 

15 FIGS. lOA and lOB are views for explaining coordinate conversion in the step 

516 shown in FIG. 6. 

FIG. 11 is a view showing a data structure of the hair-style database shown in 

FIG. 1. 

FIG. 12 is a view for explaining interpolation processing using group 
20 information included in hair data. 

FIGS. ISA and 13B are views for explaining coordinate axis rotation in 
coordinate conversion. 

FIGS. 14A and 14B are views for explaining coordinate axis shift in coordinate 
conversion. 

25 FIGS. 15A and 15B are views for explaining of mapping to the three- 

dimensional model of a head part with an arbitrary changed. 

FIG. 16 is a view showing a columnar coordinate system when hair-style data 
is generated. 

FIG. 17 is a view showing a structural example of a personal computer which 
30 reahzes an image processing apparatus. 
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DETAILED DESCRIPTION OF THE INVENTION 
A hair synthesis system to which the present invention is appHed will be 
explained with reference to FIG* 2. This hair synthesis system includes: an image 
pickup device 1 which picks up an image of a neck part (the part above the neck, 
including the neck) of a person; an image processing device 2 for generating a three- 
dimensional shape model of the neck part, based on the image of the picked-up neck 
part, synthesizing a CG of an ^bitrary hair style, and generating a hair-style-matched 
image; and a display device 9 for displaying the generated hair-style-matched image. 
If this hair synthesis system is installed in a beauty salon, a barber's shop, or the like, it 
can be used to assist customers to select hair styles or so. 

The image pickup device 1 of the hair synthesis system combines a three- 
dimensional shape measurement device such as a stereo camera, a imge finder, or the 
like, and a two-dimensional image input device such as a CCD (Charge Coupled 
Device) or the hke. In case of a stereo camera, the three-dimensional measurement 
device of the image pickup device 1 picks up an image of the head part of a person (a 
customer of a beauty salon or the like) as a target object, several times from a different 
kinds of direction, and performs predetermined image processing on the image, to 
obtain a three-dimensional shape of the face part (which is a part obtained by 
removing a hair part from the neck part). In case of a range finder, the neck part of the 
person as the target object is subjected to laser scanning, thereby measuring the 
distance, to obtain a three-dimensional shape of the neck part. The two-dimensional 
input device of the image pickup device 1 picks up an image of the neck part of the 
person as the target object, to obtain a texture corresponding to the three-dimensional 
shape of the face part. 

In general, the three-dimensional shape measurement device of the image 
pickup device 1 cannot obtain three-dimensional shape data with respect to a part of 
dark color which absorbs light from an object, e.g., a part such as a head part having 
black hair or metal which reflects light. Therefore, the unage pickup device 1 outputs 
three-dimensional shape data of the part such as the face or the hke, which will be 
hereinafter called a face part, i.e., the other part of the head part than the hair, and 
texture image data associated with the three-dimensional shape data of the face part. 
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A number of three-dimensional shape measurement devices and two- 
dimensional image input devices may be used for constructing the image pickup 
device 1, in order to measure the neck part of the person as an object (which will be 
described hereinafter as a customer) from many directions. 
5 The image processing device 2 is constructed by: a three-dimensional head part 

reconstruction processing section 3 for synthesizing three-dimensional shape data read 
from a head-part database 6 with three-dimensional shape data inputted from the 
image pickup device 1, to construct a three-dimensional model of the neck part; a hair 
style mapping processing section 4 for mapping hair-style data read from the hair-style 

10 database 5 over the three-dimensional model of the neck part; a rendering section 5 for 
rendering the three-dimensional model of the neck part on which the hair-style data 
has been mapped, thereby to generate hair-style-matched image data; a head-part 
database 6 in which three-dimensional shape data of a number of head parts are 
previously recorded; a hair-style database 7 in which a number of hair-style data items 

15 are previously recorded; and a controller 8 which is operated by a user to instruct 
selection of three-dimensional shape data of a head part stored in the head-part 
database 6, a hair-style data item stored in the hair-style database 7, or the like. 

The hair-style data stored in the hair-style database 7 is edited and generated 
with use of a three-dimensional graphic modeler or the like. Therefore, even a hair 

20 style which is actually difficult to make up can be generated and stored. The data 
structure of the hair-style database 7 will be described later with reference to FIG. 11. 
The display device 9 is constructed by a LCD (Liquid Crystal Display) or CRT 
(Cathode Ray Tube) and the like, and displays a hair-style-matched image inputted 
from the image processing device 2. 

25 Next, the operation of the hair synthesis system will be explained with 

reference to the flowchart of FIG. 3 and FIG. 4. In the step SI, the image pickup 
device 1 picks up an image of the a neck part of a customer as a target object or laser- 
scans the neck part, to measure its three-dimensional shape, and outputs three- 
dimensional shape data and texture image data of the neck part of the customer to the 

30 image processing device 2. 

The measurement method of measuring the three-dimensional shape may be 
any method; e.g., a method of using a stereo camera, a method of using a range finder. 
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or the like. As described previously, however, the three-dimensional shape 
measurement device of the image pickup device 1 cannot obtain three-dimensional 
shape data of the hair part. Therefore, three-dimensional shape data and texture image 
data (shown as measurement data in which a texture image is appHed to three- 
5 dimensional shape data of the face part in FIG. 4) of the face part of the customer is 
outputted by the processing in the step SI. If a number of three-dimensional shape 
measurement devices and two-dimensional image input devices are provided in the 
image pickup device 1, a number of sete each including three-dimensional shape data 
of the face part and texture image data are outputted. 
10 In the step S2, the three-dimensional head part reconstruction processing 

section 2 of the image processing device 2 constructs a three-dimensional shape model 
of a head part of a customer by using measurement data of the face part inputted from 
the image pickup device 1, i.e., the three-dimensional shape data and texture image 
data. 

15 Specifically, as shown in FIG. 4, head-part data selected and read from the 

head-part database 6 is combined with the measurement data of the face part of the 
customer, to reconstruct a perfect three-dimensional model of the head part of the 
customer. The head-part database 6 is constructed by three-dimensional models of a 
number of head parts having arbifrary shapes. The head-part data is subjected to 

20 scaling and positional aligning, based on the size and direction in the measurement 
data of the face part of the customer, and is then combined with the measurement data 
of the customer. 

The combination of the head-part data piece with the measurement data piece 
of the customer may generate a new polygon between open edges in both data pieces, 
25 or an open edge in one data piece may be changed to fit an open edge of the other data 
piece. 

The three-dimensional shape of the head part of the customer thus constructed 
is expressed, for example, as a triangle-polygon model. In the three-dimensional 
model of the head part of the customer thus constructed, hair is removed from the real 
30 image of the customer as shown in FIG. 4. 

In the step S3, operation of selecting hair-style data stored in the hair-style 
database 7 by a user (the customer or the like) is received by the controller 8. The 



10 



selected hair-style data is read out from the hair-style database 7, and is supplied to the 
hair style mapping processing section 4. 

In this hair style selection, icons, thumbnails, or the like indicating hair styles 
are displayed on the display device 9 to invite the user to make a selection. The 
5 selection may be done by moving a cursor or the like on the screen of the display 
device 9 or by inputting a number or the like designating a hair style. 

In the step S4, the hair style mapping processing section 4 of the image 
processing device maps the supplied hair-style data on the three-dimensional mode of 
the head part of the customer supplied from the three-dimensional head part 
10 reconstruction processing section 3, and outputs the result to the rendering part 5. In 
the processing of mapping the hair-style data, hair data corresponding to each one of 
the hairs, which is included in the hair-style data, is mapped on the surface of the 
three-dimensional model of the head part, in accordance with the shape of the three- 
dimensional model of the head part, by the same manner as taken for texture mapping. 
15 Details of the mapping processing will be described later with reference to FIGS. 5 and 
6. 

In the step S5, the rendering section 5 renders the three-dimensional model of 
the head part of the customer, on which the hair-style data has been mapped, to 
generate hair-style-matched image data as shown in FIG. 4, and outputs it to the 

20 display device 9. The method of rendering adopts texture mapping processing for a 
polygon model with respect to the face part of the three-dimensional model, and also 
rendering processing for a curvature model with respect to the hair part. The rendering 
processing for a curvature model is disclosed in details in, for example, "Simulation 
based on a motion model of a string-like object and CG" by Kobayashi, Morishima, 

25 and Harashima, Shingakugihou, PRU90- 1 27, pp. 1 5-20, 1991. 

The display device 9 displays an image corresponding to the inputted hair- 
style-matched image data. The controller 8 invites a user (a customer or the like) to 
input whether or not the user is satisfied with the hair-style-matched image displayed 
on the display device 9. 

30 At this time, the user can operate the controller 8 to watch the hair-style- 

matched image while changing the viewpoint position and the illumination position. If 
the viewpoint position and the illiunination position are changed, the rendering section 
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5 executes again the rendering processing on the three-dimensional model of the head 
part and the hair-style data, so as to correspond to the changed condition (the 
viewpoint position and the illumination position) and outputs the result to the display 
device 9. 

5 In the step S5, if an input indicating that the user is not satisfied is received, the 

processing returns to the step S3, and the later processing is repeated. In the step S5, if 

it is determined that an input indicating the user's satisfaction is received from the user, 
this processing is terminated. 

Next, an explanation will be made of the hair-style mapping processing based 
10 on the hair-style mapping processing section 4 in the step S4. Beforehand, a structural 
example of the hair-style mapping processing section 4 will be explained with 
reference to FIG. 5. 

The hair-style data read out section 21 of the hair-style mapping processing 
section 4 reads hair-style data selected with use of the controller 8, from the hair-style 

1 5 database 7, and outputs it to the mapping position calculation section 22. The mapping 
position calculation section 22 calculates the position on the three-dimensional model 
where hair data (corresponding to a hair) forming part of the hair-style data should be 
mapped. A hair data interpolation processing section 23 obtains hair data to be 
mapped, by interpolation from the peripheral hair data around the hair-style data. The 

20 coordinate conversion section 24 maps the interpolated hair data by coordinate 
conversion from the object coordinate system of the hair data to the object coordinate 
system of the three-dimensional model of the head part. 

Next, details of the hair style mapping processing of the hair style mapping 
processing section 4 will be explained with reference to the flowchart shown in FIG. 6. 

25 The mapping position calculation section 22 of the hair style mapping processing 
section 4 has already been supplied with hair-style data from the hair-style data read 
out section 21. 

The hair-style data supplied to the mapping position calculation section 22 is 
obtained by two-dimensionally arranging hair data approximated to an object having a 
30 three-dimensional shape, as shown in FIG. 7. 

In the example of FIG. 7A, every hair is approximated to a special curve such 
as a three-dimensional B-spline curve or the like. In the example of FIG. 7B, every 
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hair is approximated to a polygon model. The addresses in the two-dimensional array 
correspond to mapping coordinate values supplied to the three-dimensional model of 
the head part on which the hair style is mapped, like the texture mapping processing. 
For example, if a mapping coordinate (u, v) = (0.1, 0.1) is given to a vertex of a 
predetermined polygon of the three-dimensional model of the head part, the address on 
the hair-style data array of the hair data which is mapped on the vertex is O.lumax, 
O.lVmax. Where, the Umax is the number of data pieces in the u-direction on the hair- 
style data array, and v^ax is the number of data pieces in the v-direction on the hair- 
style data array. 

The hair-style data is defined by arranging the hair data having an arbitrary 
three-dimensional shape into a two-dimensional array. Therefore, any arbitrary hair 
style can be expressed, and it is possible to generate hair-style data of a hair style 
which is difficult to cut. A method of generating hair-style data will be described later 
with reference to FIG. 16. 

At first, in the step SI 1, the mapping position calculation section 22 initializes 
the count value v in the longitudinal direction of the hair-style data array to 0.0. In the 
step S12, the count value u in the lateral direction of the hair-style data array is 
initialized to 0.0. 

In the step SI 3, the mapping position calculation section 22 calculates the 
position on the three-dimensional model of the head part where hair data expressed by 
the address (u, v) of the hair-style data array is mapped. 

FIG. 8 shows an arbitrary triangle polygon on the three-dimensional model of 
the head part. The vertexes A, B, and C of the triangle polygon respectively have 
attribute vectors Aattr, Battr, and Cattr which express attributes of the three-dimensional 
model of the head part. 

The attribute is constructed by an object coordinate value of the three- 
dimensional model of the head part, local coordinate axes at a vertex, and a mapping 
coordinate value of hair. Where vectors which form two arbitrary edges of the triangle 
polygon ABC are vectors VI and V2, the vectors VI and V2 can be expressed by the 
next expression (1) with use of the attribute vectors of respective vertexes. 

VI = Battr -Aattr 

V2 = Cattr - Aattr (1) 
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Considering that the vectors VI and V2 are the coordinate axes of the triangle 
polygon ABC, an arbitrary point Ph on the triangle polygon ABC can be expressed by 
the next expression (2). 

Ph = Aattr + aVl + bV2 ... (2) 
5 At this time, as shown in FIG. 8, coefficients a and b are defined in a manner 

that lengths between an arbitrary point Ph and points at each of which a line parallel to 
one axis and extended from the arbitrary point crosses the other axis are normahzed by 
the lengths of axial vectors, respectively. 

Where the mapping coordinate values of the vertexes A, B, and C are 
10 respectively (ua, Va), (ub, Vb), and (Uc, Vg), attention is paid to the point Ph expressed by 
the expression (2), in relation to the elements of the mapping coordinate values. The 
next expression (3) then exists. 

Uh = Ua + a (Ub + Ua) + b (Uc - Ua) 

Vb = Va + a(Vb-Va) + b(Ve-Va)... (3) 

1 5 The expression (3) exists with respect to an arbitrary point in the (U,V) space. 

If the counter value u = Up and the counter value v = Vp, the condition for 
satisfying that the point Ph on the three-dimensional model of the head part where hair 
data should be mapped exists inside the triangle polygon is expressed by the next 
expression (4). 

20 a + b<l... (4) 

That is, in the arbitrary triangle polygon of the three-dimensional model of the 
head part, if the coefficients a and b obtained by substituting Uh = Up and Vh = Vp in the 
expression (3) satisfy the expression (4), the point Ph exists inside the triangle polygon. 
Therefore, the hair data expressed by (up, Vp) on the hair-style data array is mapped on 

25 the triangle polygon. Accordingly, the mapping position of the hair data on the three- 
dimensional model of the hair data is calculated by using the expression (2) in the 
mapped triangle polygon. 

Returning to FIG. 6, in the step S14, the mapping position calculation section 
22 determines whether or not the triangle polygon of the three-dimensional model of 

30 the head part to be mapped is a hair generation region. This determination is carried 
out, based on a flag (positive or negative) which indicates whether or not hair data 
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added to each triangle polygon of the three-dimensional model of the head part should 
be mapped. 

If the mapping flag of the triangle polygon including the mapping position 
obtained by the processing in the step S13 is positive, the triangle polygon is 
determined to be a hair generation region, and the processing goes to the step S15. 

In the step SI 5, the hair data interpolation processing section 23 interpolates 
hair data, which is expressed as (Up, Vp) and should be mqjped, with use of hair data 
stored in the hair-style data array. The interpolation method may use an average value 
of four close points or may use a summation average value of four close points. 
Alternatively, the closest point may be directly used without interpolation. 

FIG. 9 shows a method of using a summation average value from four close 
points. Supposing now that the counter value u = Up and the counter value v = Vp, the 
hair-style data array address of the hair data to be mapped is Pa = (mp, np) = (Up Umax, Vp 
Vmax)- The hair data at the point Pa is Hp, and the close 4 points are PI to P4 as shown 
in FIG. 9. The hair data pieces at the point PI to P4 are respectively HI to H4. 

In the summation average method from 4 close points, the hair data Hp to be 
mapped is interpolated from the data of the 4 close points with use of the next 
expression (5). 

Hp = dm (dn H2 + (1-dn) HI) + (1-dm) (dn H4 + (1-dn) H3)) ... (5) 

If the hair data is a special curve such as a three-dimensional B-spline curve or 
the hke as shown in FIG. 7A, the interpolation processing of the hair data is obtained 
by interpolation of data of each control point. Meanwhile, if the hair data is a polygon 
model as shown in FIG. 7B, the interpolation processing of the hair data is obtained by 
interpolation between respective polygon vertex data pieces. In the expression (4), dm 
is the decimal part of mp, and dn is the decimal part of np. 

Returning to FIG. 6, in the step SI 6, the coordinate conversion section 24 
coordinate converts hair data subjected to interpolation processing, from the object 
coordinate system of the hair data into the object coordinate system of the three- 
dimensional model of the head part. 

The concept of the coordinate conversion will be explained with reference to 
FIGS. lOA and lOB. Suppose that an arbitrary hair data piece on the hair-style data 
array is H. The hair data H is expressed as (I, J, K) according to the object coordinate 
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system of the hair data. The object coordinate system (I, J, K) of the hair data may be 
arbitrarily given in case of a three-dimensional coordinate system in which the start 
point of the hair data H is an origin. For example, this is an orthogonal coordinate 
system in which the lateral direction of the hair-style data array is I, the longitudinal 
5 direction thereof is J, and a direction perpendicular to these directions is K. Thus, the 
start point of the hair data H is expressed as (I, J, K) = (0, 0, 0), and the other control 
points are expressed by relative positions from the start point of the hair data. 

Meanwhile, the three-dimensional model of the head part is expressed as an 
object coordinate system (X, Y, Z). The object coordinate system of the three- 

10 dimensional model of the head part is generally a three-dimensional orthogonal 
coordinate system, e.g., an orthogonal coordinate system in which the direction toward 
the right ear from the gravity center position of the head part as an origin is X, the 
direction toward the top of the head is Y, and the direction toward the nose is Z, 

The coordinate conversion section 24 converts the hair data H expressed by the 

15 object coordinate system (I, J, K) of the hair data into a local coordinate system (T, S, 
N) at the mapping position Ph of the three-dimensional model of the head part obtained 
in the step SI 3. The local coordinate system (T, S, N) at this point Ph can be arbitrarily 
determined in case of the three-dimensional coordinate system using the point Ph as an 
origin. For example, it is an orthogonal coordinate system in which N is the vector in 

20 the normal line direction at the point Ph, T is the vector in the tangent direction, which 
is obtained as a vector product of the vector in the radial direction toward the point Ph 
from the center axis (Y-axis in FIG. 10) in the longitudinal direction of the three- 
dimensional model of the head part and the vector N in the normal line direction, and S 
is the vector perpendicular to the normal line direction N and the tangent direction T. 

25 Suppose that the local coordinate system at each vertex of the three- 

dimensional model of the head part is previously given in the head-part database. 
Therefore, Uke the calculation of the mapping position, the local coordinate system (T, 
S, N) at the point Ph is calculated with use of the expression (2) in the triangle polygon 
to be mapped. Supposing that data at the object coordinate system (I, J, K) of an 

30 arbitrary conti-ol point Q of the hair data H is (i, j, k), the control point C, is expressed 
by the next expression (6) in the local coordinate system (T, S, N) at the point Ph. 
Ci = iT+jS+kN ...(6) 



16 



Where, the unit vectors of the local coordinate system (T, S, N) respectively 
have values of the object coordinate system (X, Y, Z) of the three-dimensional model 
of the head part. 

Therefore, the control point Ci expressed by the expression (6) is expressed by 
5 the object coordinate system (X, Y, Z) of the three-dimensional model of the head part. 

Returning to FIG. 6, in the step SI 7, the mapping position calculation section 
22 determines whether or not the counter value u exceeds 1.0. If the counter value u is 
not determined to exceed 1.0, the processing goes to the step SI 8. In the step SI 8, the 
mapping position calculation section 22 increase the counter value u by an increment 
1 0 value du. The increment value du indicates how long pitch hair data stored in the hair- 
style data array is mapped in the lateral direction. 

Thereafter, the processing returns to the step S13, and the later processing is 
repeated. If it is determined in the step S17 that the counter value u exceeds 1.0, 
mapping processing in the lateral direction has reached the end of one line of the hair- 
15 style data array, so the processing goes to the step SI 9. 

In the step SI 9, the mapping position calculation section 22 determines 
whether or not the counter value v exceeds 1.0. If the counter value v is determined as 
not exceeding 1.0, the processing goes to the step S20. In the step S20, the mapping 
position calculation section 22 increases the counter value v by an increment value dv. 
20 The increment value dv indicates how long pitch hair data stored in the hair-style data 
array is mapped in the longitudinal direction. 

Thereafter, the processing returns to the step S12 and the later processing is 
repeated. In the step SI 9, if the counter value v is determined as exceeding 1.0, 
mapping processing is completed with respect to all lines. The hair mapping 
25 processing is terminated, and the processing goes to the step S5 shown in FIG. 3. 

In the step SI 4, if the mapping flag of the triangle polygon including the 
mapping position obtained by the processing in the step S13 is negative, the triangle 
polygon is not determined to be a hair generation region, and the processing of the 
steps SI 5 and S16 is skipped. 
30 By thus specifying a hair generation region with use of a mapping flag, 

mapping can be effected only on an arbitrary region of the three-dimensional model of 
the head part, without mapping all the hair data pieces that construct the hair-style data 
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read out fi-om the hair-style database 7. Therefore, a hair-style such as a so-called 
Machican cut, a star cut, or the like can be easily generated. 

Also, according to this hair-style mapping processing, each of the hair-style 
data pieces stored in the hair-style database 7 is expressed by the object coordinate 
5 system of hair and can be mapped on a three-dimensional model of a head part having 
any arbitrary shape, without depending on the shape of the human head part. 

Next, the data structure of the hair-style database 7 will be explained with 
reference to FIG. 11. The hair-style database 7 is constructed by a number of hair- 
style data pieces. Each hair-style data piece is of a two-dimensional array as described 
10 above, and at the top, the number of data pieces Umax in the lateral direction and the 
mmiber of data pieces Vmax in the longitudinal direction. Following them, hair data 
pieces for (umax x Vnjax) are described. 

In some cases, group information (described later) is described at the top of the 
hair data piece. Subsequently, a number of control point data pieces are described, in 
15 the case where hair data is approximated to a three-dimensional parametric curve or 
the like, the control point data pieces are expressed by the hair object coordinate 
system (I, J, K). 

The hair data may be three-dimensional polygon data. In this case, a number 
of polygon vertex data pieces are described as hair data. 

20 The group information described in the hair data can be used for hair data 

interpolation processing in the step SI 5 of the hair-style mapping processing (FIG. 6) 
described above. 

With reference to FIG. 12, explanation will be made of the hair data 
interpolation processing where group information is supplied. Where the coimter 

25 value u = Up and the counter value v = Vp are given, the address on the hair-style data 
array of hair data to be mapped is Pa = (mp, np) = (up u^ax, Vp Vmax)- 

Suppose that the hair data at the point Pa is Hp, four close points are points PI 
to P4 as shown in FIG. 12, and hair data pieces at the points PI to P4 are respectively 
HI to H4. Further, 1 is described as group information in the hair data piece HI, and 2 

30 is described as group information in each of the hair data pieces H2 to H4. That is, in 
this example, only the hair data piece HI belongs to a group 1, and the other hair data 
pieces H2 to H4 belong to a group 2. If group information is given, only the hair data 
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pieces that belong to one same group are used to perform interpolation, in the hair data 
interpolation processing. 

More specifically, among the four close points of the point Pa which is mapped 
at first, the closest point if obtained and the group information thereof is obtained. In 
5 case of FIG. 12, the closest point is the point P3. Next, among the four close points PI 
to P4, the number of close points that belong to the same group as that of the closest 
point is obtained. 

If no other point among the four close points PI to P4 than the close point 
belongs to the same group as that of the closest point, the hair data piece of the closest 
10 point is directly used also for the point Pa. 

If two close points including the closest point among the four close points PI to 
P4 belong to the same group as that of the closest point, a linear interpolation value 
between the two points is used for interpolation. 

If three close points including the closest point among the four close points PI 
15 to P4 belong to the same group as that of the closest point, i.e., in the case shown in 
FIG. 12, an interpolation value is calculated by the next expression (7). 

Hp = (l-r)H32 + rH34 

H32 = dnH2 + (l-dn)H3 

H34 = dnH4 + (l-dn)H3 
20 r = (mp - m32)/(m34 - m32) . . . (7) 

In this expression, dn is the decimal part of Up, m32 is the m-address at the 
point P32, and m34 is the m-address at the point P34. 

By thus describing group information in hair data, for example, it is possible to 
express even a hair style having parting of hair where interpolation will cause a 
25 problem. That is, if grouping at the boimdary of parting is performed, hair data pieces 
in both sides of the boundary can be used to prevent generation of unnatural hair, in 
case where four close points spread over the boundary. 

Next, with reference to FIGS. 12 and 13, explanation will be made of an 
applicative example of the coordinate conversion in the processing in the step S16 in 
30 the hair-style mapping processing (FIG. 6) described above. 

In the coordinate conversion processing in the step SI 6, unevenness in a hair 
style can be expressed with use of random numbers. FIG. 13A shows a state where 
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unevenness in a hair style is not expressed while FIG. 13B shows an example of a state 
in which unevenness in a hair style is expressed by rotating the local coordinate system 
with use of a random number in the coordinate conversion. 

Specifically, the local coordinate system (T, S, N) of the head part is rotated 
5 about an axis in correspondence with a generated random number where the axis is N. 
Thereafter, the result is converted into the object coordinate system (X, Y, Z) of the 
head part model. By this processing, the directions in which respective hairs grow can 
be changed dehcately, without changing the positions where hair data should be 
mapped. 

10 FIG. 14A shows a state in which unevenness in a hair style is not expressed, 

while FIG. 14B shows an example in which unevenness in a hair style is expressed by 
changing mapping positions by random numbers in the coordinate conversion. 

Specifically, the position of Ph as the origin of the local coordinate system (T, 
S, N) is shifted to in correspondence with a generated random number, and is 
15 thereafter converted into the object coordinate system (X, Y, Z) of the head part 
model. By this processing, the position where the hair grows can be changed 
dehcately without changing the hair style. 

The technique of rotating the axis of the coordinate system to be converted use 
the random number and/or moving its origin to express natural unevenness in a 
20 synthesized image can be applied to other cases than the synthesis of hair images. 

Meanwhile, in this hair synthesis system, hair-style data can be mapped after 
changing the three-dimensional model of the head part into an arbitrary shape by the 
three-diniensional head part reconstruction processing section 3. FIG. 15 A shows a 
state in which the three-dimensional model of a head part is not changed, while FIG. 
25 15B shows a state in which the three-dimensional model of the head part is magnified 
1.5 times in the Y-axis direction. 

Magnification/minification processing for a three-dimensional model of a head 
part is achieved by affine conversion processing. At this time, similar affine 
conversion processing is performed also on the local coordinate system (T, S, N) 
30 assigned to an arbitrary polygon vertex of the three-dimensional model of a head part. 

As shown in FIG. 15B, in the case of magnifying the three-dimensional model 
of the head part for 1.5 times in the Y-axis direction, vectors of T, S, and N are 
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multiplied by a conversion matrix for 1.5 times magnification in the Y-axis direction, 
to obtain a local coordinate system (T', S', N') expanded in the longitudinal direction. 
By mapping hair data H on the local coordinate system (T', S', N') thus deformed, the 
hair style is magnified 1.5 times in the Y-axis direction, into a shape matched with the 
5 head part. By this processing, same hair-style data can be mapped on the three- 
dimensional model of a head part, which is deformed into an arbitrary shape. Next, 
generation of hair-style data will be explained with reference to FIG. 16. FIG, 16 
shows a columnar coordinate system which surrounds a head part. In the columnar 
coordinate system, the axis penetrating the center of a column is taken as the Y-axis, 

10 the distance from the Y-axis to a plane perpendicular to the Y-axis is r, and the angle 
from a reference direction on this plane is 0. 

An arbitrary point Ph on the surface of the head part can be expressed as Pj, = 
(Ypj Tpj 0p) with use of the columnar coordinate system. Further, if the line defined by 
Y = yp and 0 = 9p of the columnar coordinate system always has one contact point with 

1 5 the surface of the head part, the arbitrary point Ph on the surface of the head part can be 
expressed by (yp, 9p). 

That is, if the shape of the surface of the head part is always convex toward the 
circumferential direction, the above condition (the line defined by Y= yp and 0 = 0p of 
the columnar coordinate system always has one contact point with the surface of the 

20 _head part) is satisfied. 

Considering the scalp part excluding the face, the shape of the surface of the 
head part is generally considered to be always convex toward the circumferential 
direction. Therefore, the arbitrary point Ph on the scalp part of the head-part model 
can be expressed by Ph = (yp, 0p). 

25 Creation of hair-style data can be relatively easily carried out by using a system 

disclosed in "Development of a hair design system using tuft models" by Kishi, 
Saegusa, and Morishima, in Shingakugihou, MVE97-102, pp. 67-74, 1998. In the 
system, hair-style data is prepared by editing hair data generated on a given head-part 
model. Hair data outputted from this system is expressed by the object coordinate 

30 system of the given head-part model. 

The hair database 7 in the present invention records hair-style data which is 
stored in a two-dimensional array by mapping the hair data on a space expressed by (y. 



21 



0) of a columnar coordinate system. For example, the following expression (8) is used 
for conversion from the (y, 0) space into the (u, v) space of the two-dimensional array 
as shown in FIG. 7. 
u = 0p / 27t 

5 V = (yp-y™„) / (ymax-y™!.) ... (8) 

In this expression, ymm is the smallest value of the Y-axis of the columnar 
coordinate system of the surface of the head part, and ymax is the largest value of the Y- 
axis of the colimmar coordinate system of the surface of the head part. 

The conversion from the (y, 0) space into the (u, v) space may be of non-linear. 
10 For example, sampHng near the top part of the head can be prevented from being 
coarse, as long as the conversion is ranged such that the increment in the v-direction 
becomes smaller as it comes closer to the head top. 

As described above, hair-style data consisting of hair data converted into the 
(u, v) space is recorded in the hair-style database 7. 
15 The present invention is also applicable to the case of expressing a string-like 

object other than hair in CG. 

The series of processing in the image processing apparatus 2 as described 
above can be executed by both hardware and software. In case of executing the series 
of processing by software, the soflwa-e can be installed from a recording medium into 
20 a computer in which the program constituting the software is incorporated in dedicated 
hardware or a general-purpose computer which can execute various functions 
computer by installing various programs. 

FIG. 17 shows a structural example of a personal computer which embodies the 
image processing apparatus 2. This personal computer includes a CPU (Central 
25 Processing Unit) 41. The CPU 41 is connected with an input/output interface 45 
through a bus 44. The input/output interface 45 is connected with: an input section 46 
constructed by an input device such as a keyboard, a mouse, or the like; an image 
output section 47 for outputting a hair-style-matched image to the display device 9; a 
storage section 48 constructed by a hard-disc drive for storing a program, various data, 
30 and the like; a commimication section 49 for communicating data through Internet or 
the like; and a drive 50 for reading/writing data with respect to a recording medium 
such as a magnetic disc 51, a semiconductor memory 54, or the like. The bus 44 is 
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connected with a ROM (Read Only Memory) 42 and a RAM (Random Access 

Memory) 43. 

An image processing program which lets the personal computer execute 
operation as the image processing apparatus 2 is supplied in the form that it is stored in 
5 the magnetic disc 51 or semiconductor memory 54, to the personal computer. It is 
read by the drive 50 and installed in the hard disc drive included in the storage section 
48. The image processing program installed in the storage section 48 is loaded from 
the storage section 48 to the RAM 43 and executed, in accordance with instructions 
from the CPU 41 corresponding to commaads from the user. 

10 In the present invention, the step of describing the program recorded in the 

recording medium includes not only processing which is carried out based on time in 
accordance with a described order but also processing which is executed in parallel or 
individually although it is not always performed based on time. 

Also, in the present invention, the term of "system" includes the entire 

1 5 apparatus constructed of a number of apparatuses and/or devices. 
Industrial Applicability 

As described above, according to the present invention, the hair-style is stored, 
stored hair-style data is read out, and hair data corresponding to a hair and included in 
the hair-style data is mapped at a predetermined position of a three-dimensional shape 

20 image of a head part. Therefore, a hair style consisting of hairs which are expressed 
by CG can be changed so as to correspond to a number of different head shapes. Also, 
according to the present invention, stored hair-style data is read out, and hair data 
corresponding to a hair and included in the read out hair-style data is mapped at a 
predetermined position on a three-dimensional shape image of a head part. Based on 

25 the mapping results, a hair-style-matched image is generated. Therefore, a hair style 
consisting of hairs which are expressed by CG can be changed so as to correspond to a 
number of different head shapes. 

Although the present invention has been described with reference to specific 
embodiments, those of skill in the art will recognize that changes may be made thereto 

30 without departing from the spirit and scope of the invention as set forth in the hereafter 
appended claims. 
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ABSTRACT OF THE DISCLOSURE 
The image processing apparatus changes a hair style of hairs which are 
expressed by computer graphics ("CG"), so as to correspond to a number of different 
head part shapes. The apparatus includes: a three-dimensional head part 
5 reconstruction processing section for synthesizing three-dimensional shape data of a 
face part inputted from an image pickup device with three-dimensional shape data of a 
head part read out from a head-part database, to construct a three-dimensional shape 
model of the neck part; a hair-style mapping processing section for mapping hair data 
read out from a hair-style database on the three-dimensional shape model of the neck 
10 part; and a rendering section for rendering the three-dimensional shape model of the 
neck part on which the hair-style data is mapped, to generate hair-style-matched image 
data. The hair-style database stores a number of pieces of hair-style data. 
In the Claims; 

Please amend Claims 1, 5-11, 15-21, and 25-30 as follows: 
15 1 . (Once amended) An image processing apparatus for synthesizing a hair 

image with a three-dimensional shape image of a head part, to generate a hair-style- 
matched image, comprising: 

a storage part for storing a hair-style data piece constructed by a 
plurality of hair data pieces arranged on a two-dimensional array; 
20 a read out part for reading out the hair-style data piece stored in the 

storage part; 

a mapping part for mapping the hair data piece corresponding to a hair 
contained in the hair-style data piece read out by the read out part, at a predetermined 
position on the three-dimensional shape image of the head part; and 

25 a generation part for generating the hair-style-matched image, based on 

a mappmg result obtained by the mapping part. 

5. (Once amended) The image processing apparatus according to claim 1, 
wherein the storage part stores the hair-style data piece constructed with a plurality of 
data pieces arranged on a two-dimensional array corresponding to a projected image 

30 obtained by projecting a three-dimensional hair style expressed by a columnar 
coordinate system on a two-dimensional coordinate system. 
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6. (Once amended) The image processing apparatus according to claim 1, 
wherein an interpolation part interpolates the hair data piece at the predetermined 
position with use of a hair data piece close to the predetermined position. 

7. (Once amended) The image processing apparatus according to claim 6, 
5 wherein the interpolation part interpolates the hair data piece based on group 

information contained in the hair data piece close to the predetermined position. 

8. (Once amended) The image processing apparatus according to claim 1, 
wherein the mapping part includes a coordinate conversion part for converting the hair 
data piece expressed by a first coordinate system into data of a second coordinate 

1 0 system by which the three-dimensional shape image of the head part is expressed. 

9. (Once amended) The image processing apparatus according to claim 8, 
wherein the coordinate conversion part converts the hair data piece expressed by the 
first coordinate system into data of a local coordinate system having an origin on a 
surface of the head part, and thereafter converts the data into data of the second 

15 coordinate system by which the three-dimensional shape image of the head part is 
expressed. 

10. (Once amended) The image processing apparatus according to claim 8, 
wherein when the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 

20 image of the head part is expressed, the coordinate conversion part executes at least 
one of a coordinate axis rotation and origin shift, based on a random number. 

1 1 . (Once amended) An image processing method for synthesizing a hair 
image with a three-dimensional shape image of a head part, to generate a hair-style- 
matched image, die method comprising the steps of: 

25 storing a hair-style data piece constructed by a plurahty of hair data 

pieces arranged on a two-dimensional array; 

reading out the hair-style data piece; 

mapping the hair data piece corresponding to a hair contained in the 
read out hair-style data piece at a predetermined position on the three-dmiensional 
30 shape image of the head part; and 

generating the hair-style-matched image based on the mapping result. 
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15. (Once amended) The image processing method according to claim 11, 
wherein during the step of storing, the hair-style data piece is constructed by a plurality 
of hair data pieces arranged on a two-dimensional array corresponding to a projected 
image obtained by projecting a three-dimensional hair style expressed by a columnar 

5 coordinate system, on a two-dimensional coordinate system, and is stored. 

16. (Once amended) The image processing method accordmg to claim 11, 
further comprising the step of interpolating the hair data piece at the predetermmed 
position with use of a hair data piece close to the predetermined position. 

17. (Once amended) The image processing method according to claim 16, 
10 wherein during the step of interpolating, the hair data piece is interpolated based on 

group information contained in the hair data piece close to the predetermined position. 

18. (Once amended) The image processing method according to claim 11, 
wherein the step of mapping includes a coordinate conversion st^ of converting the 
hair data piece expressed by a first coordinate system into data of a second coordinate 

15 system by which the three-dimensional shape image of the head part is expressed. 

19. (Once amended) The image processing method according to claim 18, 
wherein the coordinate conversion step includes converting the hair data piece 
expressed by the first coordinate system into data of a local coordinate system having 
an origin on a surface of the head part, and thereafter, converting the data into data of 

20 the second coordinate system by which the three-dimensional shape image of the head 
part is expressed. 

20. (Once amended) The image processing method according to claim 18, 
wherein the step of converting the hair data piece expressed by the first coordinate 
system into the data of the second coordinate system by which the three-dimensional 

25 shape image of the head part is expressed includes executing at least one of coordinate 
axis rotation and origin shift based on a random number, in processing in the 
coordinate conversion step. 

21. (Once amended) A recording medium which records a computer- 
readable program for image processing of synthesizing a hair image with a three- 

30 dimensional shape image of a head part, to generate a hair-style-matched image, the 
program comprising: 
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a storage part for of storing a hair-style data piece constructed by a 
plurality of hair data pieces arranged on a two-dimensional array; 

a read out part for of reading out the hair-style data piece stored by 
processing in the storage part; 

a mapping part for of mapping the hair data piece corresponding to a 
hair contained in the hair-style data piece read out by the read out paxt, at a 
predetermined position on the three-dimensional shape image of the head part; and 

a generation part for of generating the hair-style-matched image, based 
on a mapping result obtained by the mapping part. 

25. (Once amended) The recording medium according to claim 21, 
wherein, during processing in the storage part, the hair-style data piece constructed 
with a pluraUty of data pieces arranged on a two-dimensional array corresponding to a 
projected image obtained by projecting a three-dimensional hair style expressed by a 
columnar coordinate system, on a two-dimensional coordinate system is stored. 

26. (Once amended) The recording medium according to claim 21, wherein 
the program further comprises an interpolation part for of interpolating the hair data 
piece at the predetermined position with use of a hair data piece close to the 
predetermined position. 

27. (Once amended) The recording medium according to claim 26, 
wherein, during processing in the interpolation part, the hair data piece is interpolated 
based on group information contained in the hair data piece close to the predetermined 
position. 

28. (Once amended) The recording mediirai according to claim 21, 
wherein, processing in the mapping part includes a coordinate conversion step of 
converting the hair data piece expressed by a first coordinate system into data of a 
second coordinate system by which the three-dimensional shape image of the head part 
is expressed. 

29. (Once amended) The recording medium according to claim 28, 
wherein, during processing in the coordinate conversion part, the hair data piece 
expressed by the first coordinate system is converted into data of a local coordinate 
system having an origin on a surface of the head part, and thereafter, the data is 
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converted into data of the second coordinate system by which the three-dimensional 
shape image of the head part is expressed. 

30. (Once amended) The recording medium according to claim 28, 
wherein, when the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 
image of the head part is expressed, at least one of coordinate axis rotation and origin 
shift is executed, based on a random number, during processing in the coordinate 
conversion part. 
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REMARKS 



The present amendment makes editorial changes and corrects typographical 
errors in the specification, which includes the Abstract, in order to conform the 
specification to the requirements of United States Patent practice. No new matter is 
added thereby. Attached hereto is a marked-up version of the changes made to the 
specification by the present amendment. The attached page is captioned " Version 
With Markings To Show Changes Made". 

The present amendment is intended for clarification pxirposes only and not for 
substantial reasons related to patentability pursuant to 35 U.S.C. §§101, 102, 103 or 
112. Indeed, the amendment of claims 1, 5-11, 15-21 and 25-30 does not constitute an 
intent on the part of the Apphcant to surrender any of the subject matter of claims 1, 5- 
11, 15-21 and 25-30. 

Early consideration on the merits is respectfully requested. 
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VERSION WITH MARKINGS TO SHOW CHANGES MADE 
In the Specification: 

SPECIFICATION 

DESCRIPTIO N TITLE OF THE INVENTION 

5 IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND 
RECORDING MEDIUM 
BACKGROUND OF THE INVENTION 

T e ohmoal Fi e ld 

The present invention relates to an image processing apparatus, an image 
10 processing method, aad a recording mediimi , and more Gpecifically to an imag e 
processing apparatus, an image processing method, and a recording medium which are 
suitable for the case of synthesizing a hair image expressed by computer graphics with 
a three-dimensional shape image of a head par t is r e moving a hair . 
Background Art 

15 Conventionally, as a method for expressing human hair by computer graphics 

(hereinafter abbreviated as CG), it is useful to take a method in which an entire hair 
style is approximated to a polygon model, as shown in FIG. lA, and a texture of hairs 
is adhered to the model, or a method in which rendering is performed with use of an 
anisotropic reflection model which has different reflection intensities respectively with 
20 respect to light source positions or viewpoint positions. 

These methods are described in "Hair expression using an anisotropic 
reflection model" by Yamana and Suenaga, Singakugihou, PRU87-3, pp. 115 to 120, 
1989. In these methods, expression of a hair style is just the shape of a polygon model 
itself, and the hair style can be changed by changing the shape of the polygon model4s 

A proposal has been made for a method of expressing hair by a particle system. 
In the particle system, when a large quantity of particle objects are dealt with, 
characteristics such as weight and the like of each particle are given as parameters, and 
particles are generated under a certain rule. Thus, a large quantity of particle objects 
30 can be expressed easily. 
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Meanwhile, the present apphcant h^ proposed a method of approximating 
every on e of hairs h air of a hairstyle t o a curve, as shown in FIG. IB, in Japanese 
Patent Application No. 10-164545, for example. This method is successful in natural 
expression of a hair style image. 
5 A method of designing a hair style by editing together a plurality of hairs 

approximated to a curve is described in "Development of a hair design system using 
tuft models" by Kishi, Saegusa, and Morishima, in Shingakugihou, MVE97-102, pp. 
67-74,1998. 

It is difficult for the method of approximating the entire hair style to a polygon 
10 model to express a comphcated hair style. In this method, movements (flows) of hair 
cannot be expressed. 

On the other side, the method of expressing hair by a particle system and the 
method of approximating every on e of hairs h air t o a geometric figure can express any 
arbitrary hair style and movements of hair. However, in the method of expressing hair 
15 by the particle system, parameters for particles expressing hairs must be controlled to 
control the hair style, so that intuitive control is difficult. It is therefore difficult to 
generate an intended hair style. 

In the method described in "Development of a hair design system using tuft 
models" by Kishi, Saegusa, and Morishima, in Shingakugihou, MVE97-102, pp. 67- 
20 74, 1998, an intended hair style can be generated relatively easily. This method, 
however, has a problem in that hair cannot be changed to intended hair styles 
corresponding to a plurality of different head shapes although hair corresponding only 
to one fixed head shape can be changed to an intended hair style/? 

Dir.c1or.uro of the Inventio nS UMMARY OF THE INVENTION 
25 Th e pres e nt invention has been made in view of the actual situation as 

describ e d abov e and has a n One of the objects of the present invention is changing a 
hair style consisting of hairs every of which is expressed by CG, so as to correspond to 
pfa^al -a number o f different head shapes. 

The present invention propos e d to achi e ve the abov e obj e ct p rovides an image 
30 processing apparatus for synthesizing a hair image with a three-dimensional shape 
image of a head part, to generate a hair-style-matched image, comprisin gi ncluding : a 
storage meaas -part for storing a hair-style data piece; a_read out meaas -part for reading 
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out the hair-style data piece stored in the storage meassEMt; a.niapping meaas -part for 
mapping a hair data piece corresponding to a hair and contained in the hair-style data 
piece read out by the read out meanop art., at a predetermined position on the three- 
dimensional shape image of the head part; and a^eneration meafits-parLfor generating 
5 the hair-style-matched image, based on a mapping result obtained by the mapping 

As the hair data piece, it is possible to use three-dimensional curve data which 
is constructed by a plurality n umber of control points. 

As the hair data piece, it is possible to use polygon data which is constructed 
10 by a plurality n umber of polygons. 

The storage Hteaas -part stores the hair-style data piece constructed by plural 
hair data pieces arranged on a two-dimensional array. In particular, the storage means 
part stores the hair-style data piece constructed by plural data pieces arranged on a 
two-dimensional array corresponding to a projected image obtained by projecting a 
15 three-dimensional hair style expressed by a columnar coordinate system, on a two- 
dimensional coordinate system. 

The apparatus according to the present invention further comprises includes an 
interpolation meaas -part for interpolating the hair data piece at the predetermined 
position, with use of a hair data piece close to the predetermined position. The 
20 interpolation m€>aas -part interpolates the hair data piece, based on group information 
contained in the hair data piece close to the predetermined position. 

The mapping m e ans p art i ncludes a^coordinate conversion means p art for 
converting the hair data piece expressed by a first coordinate system into data of a 
second coordinate system by which the three-dimensional shape image of the head part 
25 is expressed. 

The coordinate conversion meafis-part converts the hair data piece expressed by 
the first coordinate system into data of a local coordinate system having an origin on a 
surface of the head part, and thereafter converts the data into data of the second 
coordinate system by which the three-dimensional shape image of the head part is 
30 expressed. 

When the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 
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image of the head part is expressed, the coordinate conversion meano p art executes at 
least one of coordinate axis rotation and origin shift, based on a random number. 

Also, the present invention provides an image processing method for 
synthesizing a hair image with a three-dimensional shape image of a head part, to 
5 generate a hair-style-matched image, oomprisin | gi ncluding : a storage step of storing a 
hair-style data piece; a read out_step of reading out the hair-style data piece stored by 
processing in the storage step; a mapping step of mapping a hair data piece 
corresponding to a hair and contained in the hair-style data piece read out by 
processing in the read out_step, at a predetermined position on the three-dimensional 
10 shape image of the head part; and a generation step of generating the hair-style- 
matched image, based on a mapping result obtained by processing in the mapping step. 

In the processing in the storage step according to the present invention, the 
hair-style data piece constructed by plural h air data pieces arranged on a two- 
dimensional array is stored. 
15 Also, in the processing in the storage step, the hair-style data piece constructed 

by plural data pieces arranged on a two-dimensional array corresponding to a projected 
image obtained by projecting a three-dimensional hair style expressed by a columnar 
coordinate system, on a two-dimensional coordinate system, is stored. 

The method according to the present invention further oompris e o i ncludes an 
20 interpolation step of interpolating the hair data piece at the predetermined position, 
with use of a hair data piece close to the predetermined position. 

In processing in the interpolation step, the hair data piece is interpolated, based 
on group information contained in the hair data piece close to the predetermined 
position. 

25 The processing in the mapping step includes a coordinate conversion step of 

converting the hair data piece expressed by a first coordinate system into data of a 
second coordinate system by which the three-dimensional shape image of the head part 
is expressed. 

In processing in the coordinate conversion step, the hair data piece expressed 
30 by the first coordinate system is converted into data of a local coordinate system 
having an origin on a surface of the head part, and thereafter, the data is converted into 
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data of the second coordinate system by which the three-dimensional shape image of 
the head part is expressed. 

When the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 
5 image of the head part is expressed, at least one of coordinate axis rotation and origin 
shift is executed, based on a random nimiber, in processing in the coordinate 
conversion step. 

Further, the present invention provides a recording medium which records a 
computer-readable program for image processing of synthesizing a hair image with a 

10 three-dimensional shape image of a head part, to generate a hair-style-matched image, 
the program oomprioin p including : a storage step of storing a hair-style data piece; a 
read out.step of reading out the hair-style data piece stored by processing in the storage 
step; a mapping step of mapping a hair data piece corresponding to a hair and 
contained in the hair-style data piece read out by processing in the read out_step, at a 

15 predetermined position on the three-dimensional shape image of the head part; and a 
generation step of generating the hair-style-matched image, based on a mapping result 
obtained by processing in the mapping step. 

In the processing in the storage step, the hair-style data piece constructed by 
pteal-the hair data pieces arranged on a two-dimensional array is stored. 

20 In the processing in the storage step, the hair-style data piece constructed by 

plural data pieces arranged on a two-dimensional array corresponding to a projected 
image obtained by projecting a three-dimensional hair style expressed by a columnar 
coordinate system, on a two-dimensional coordinate system, is stored. 

According to the present invention, the program further oomprip e s i ncludes an 

25 interpolation step of interpolating the hair data piece at the predetermined position, 
with use of a hair data piece close to the predetermined position. 

In processing in the interpolation step, the hair data piece is interpolated, based 
on group information contained in the hair data piece close to the predetermined 
position. 

30 The processing in the mapping step includes a coordinate conversion step of 

converting the hair data piece expressed by a first coordinate system into data of a 
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second coordinate system by which the three-ditnensional shape image of the head part 
is expressed. 

In processing in the coordinate conversion step, the hair data piece expressed 
by the first coordinate system is converted into data of a local coordinate system 
5 having an origin on a surface of tiie head part, and thereafter, the data is converted into 
data of the second coordinate system by which the three-dimensional shape image of 
the head part is expressed. 

Wlien the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 
1 0 image of the head part is expressed, at least one of coordinate axis rotation and origin 
shift is executed, based on a random number, in processing in the coordinate 
conversion step. 

In the above-described image processing apparatus said method according to 
the present invention, hair-style data is stored, and the stored hair-style data is read out. 
15 Hair data corresponding to a hair and included in the read out_hair-style data is mapped 
at a predetermined position on a three-dimensional shape image of a head part. Based 
on the mapping results, a hair-style-matched image is generated. The stored hair-style 
data is constructed by pteal-pieces of hair data arranged on a two-dimensional array. 

Further other obj e oto of th e pr e oont invention and opeoifio advantag e s which 
20 will b e obtained from the preoent invention will be more clearly und e rotood firom th e 
following description of embodiments- A dditional features and advantages of the 
present invention are described in, and will be apparent from, the following Detailed 
Description of the Invention and the Figures. 

Brinf DnnRriptinn nf tho Dra^^nnas B RIEF DESCRIPTION OF THE FIGURES 
25 FIGS. 1-A and IB are views showing an example of conventional hair 

expression. 

FIG. 2 is a block diagram showing a structural example of a hair synthesis 
system to which the present invention is applied. 

FIG. 3 is a flowchart explaining operation of the hair synthesis system. 
30 FIG. 4 is a view for explaining the operation of the hair synthesis system, 

FIG. 5 is a block diagram showing a structural example of processing in the 
hair-style mapping processing section shown in FIG. 2. 
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FIG. 6 is a flowchart for explaining processing in the hair-style mapping 
processing section. 

FIGS. 7 A and 7B are views for explaining hair-style data in which hair data is 
two-dimensionally arrayed. 
5 FIG. 8 is a view for explaining mapping position calculation in the step S13 

shown in FIG. 6. 

FIG. 9 is a view for explaining hair data interpolation processing in the step 
S15 shown in FIG. 6. 

FIGS. lOA and lOB are views for explaining coordinate conversion in the step 
10 S16 shown in FIG. 6. 

FIG. 1 1 is a view showing a data structure of the hair-style database shown in 

FIG. 1. 

FIG. 12 is a view for explaining interpolation processing using group 
information included in hair data. 



13 15 FIGS. 13A and 13B are views for explaining coordinate axis rotation in 

1=1= coordinate conversion. 

^} FIGS. 14A and 14B are views for explaining coordinate axis shift in coordinate 

ftJ conversion. 

FIGS. 15A and 15B are views for explaining of mapping to the three- 
20 dimensional model of a head part with an arbitrary changed. 

FIG. 16 is a view showing a columnar coordinate system when hair-style data 
is generated. 

FIG. 17 is a view showing a structural example of a personal computer which 
reaMzes an image processing apparatus. 
25 Boot Mod e for Carr\ring out tho Invontion D ETAILED DESCRIPTION OF THE 

INVENTION 

A hair synthesis system to which the present invention is applied will be 
explained with reference to FIG. 2. This hair synthesis system comprises i ncludes : an 
image pickup device 1 which picks up an image of a neck part (the part above the 
30 neck, including the neck) of a person; an image processing device 2 for generating a 
three-dimensional shape model of the neck part, based on the image of the picked-up 
neck part, synthesizing a CG of an ^bitrary hair style, and generating a hair-style- 
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matched image; and a display device 9 for displaying the generated hair-style-matched 
image. If this hair synthesis system is installed in a beauty salon, a barber's shop, or 
the like, it can be used to assist customers to select hair styles or so. 

The image pickup device 1 of the hair synthesis system combines a three- 
5 dimensional shape measurement device such as a stereo camera, a range finder, or the 
like, and a two-dimensional image input device such as a CCD (Charge Coupled 
Device) or the like. In case of a stereo camera, the three-dimensional measurement 
device of the image pickup device 1 picks up an image of the head part of a person (a 
customer of a beauty salon or the like) as a target object, several times from a different 

10 kinds of direction, and performs predetermined image processing on the image, to 
obtain a three-dimensional shape of the face part (which is a part obtained by 
removing a hair part from the neck part). In case of a range finder, the neck part of the 
person as the target object is subjected to laser scanning, thereby measuring the 
distance, to obtain a three-dimensional shape of the neck part. The two-dimensional 

15 input device of the image pickup device 1 picks up an image of the neck part of the 
person as the target object, to obtain a texture corresponding to the three-dimensional 
shape of the face part. 

In general, the three-dimensional shape measurement device of the image 
pickup device 1 cannot obtain three-dimensional shape data with respect to a part of 

20 dark color which absorbs light from an object, e.g., a part such as a head part having 
black hair or metal which reflects light. Therefore, the image pickup device 1 outputs 
three-dimensional shape data of the part such as the face or the like, which will be 
hereinafter called a face part, i.e., the other part of the head part than the hair, and 
texture image data associated with the three-dimensional shape data of the face part. 

25 Ptealr -A number of three-dimensional shape measurement devices and plural 

two-dimensional image input devices may be used for constructing the image pickup 
device 1, in order to measure the neck part of the person as an object (which will be 
described hereinafter as a customer) from many directions. 

The image processing device 2 is constructed by: a three-dimensional head part 

30 reconstruction processing section 3 for synthesizing three-dimensional shape data read 
from a head-part database 6 with three-dimensional shape data inputted from the 
image pickup device 1, to construct a three-dimensional model of the neck part; a hair 
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style mapping processing section 4 for mapping hair-style data read from the hair-style 
database 5 over the three-dimensional model of the neck part; a rendering section 5 for 
rendering the three-dimensional model of the neck part on which the hair-style data 
has been mapped, thereby to generate hair-style-matched image data; a head-part 
database 6 in which three-dimensional shape data of a plurality n umber of head parts 
are previously recorded; a hair-style database 7 in which a plurality n umber of hair- 
style data items are previously recorded; and a controller 8 which is operated by a user 
to instruct selection of three-dimensional shape data of a head part stored in the head- 
part database 6, a hair-style data item stored in the hair-style database 7, or the like. 

The hair-style data stored in the hair-style database 7 is edited and generated 
with use of a three-dimensional graphic modeler or the like. Therefore, even a hair 
style which is actually difficult to make up can be generated and stored. The data 
structure of the hair-style database 7 will be described later with reference to FIG. 11. 
The display device 9 is constructed by a LCD (Liquid Crystal Display) or CRT 
(Cathode Ray Tube) and the like, and displays a harr-style-matched image inputted 
from the image processing device 2. 

Next, the operation of the hair synthesis system will be explained with 
reference to the flowchart of FIG. 3 and FIG. 4. In the step SI, the image pickup 
device 1 picks up an image of the a neck part of a customer as a target object or laser- 
scans the neck part, to measure its three-dimensional shape, and outputs three- 
dimensional shape data and texture image data of the neck part of the customer to the 
image processing device 2. 

The measurement method of measuring the three-dimensional shape may be 
any methodTi e.g., a method of using a stereo camera, a method of using a range finder, 
or the like. As described previously, however, the three-dimensional shape 
measurement device of the image pickup device 1 cannot obtained obtain three- 
dimensional shape data of the hair part. Therefore, three-dimensional shape data and 
texture image data (shown as measurement data in which a texture image is applied to 
three-dimensional shape data of the face part in FIG. 4) of the face part of the customer 
is outputted by the processing in the step SI. If pltH^^number of three-dimensional 
shape measurement devices and two-dimensional image input devices are provided in 
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the image pickup device 1, a plurality m rniber of sets each including three-dimensional 
shape data of the face part and texture image data are outputted. 

In the step S2, the three-dimensional head part reconstruction processing 
section 2 of the image processing device 2 constructs a three-dimensional shape model 
of a head part of a customer by using measurement data of the face part inputted from 
the image pickup device 1, i.e., the three-dimensional shape data and texture image 
data. 

Specifically, as shown in FIG. 4, head-part data selected and read from the 
head-part database 6 is combined with the measurement data of the face part of the 
customer, to reconstruct a perfect three-dimensional model of the head part of the 
customer. The head-part database 6 is constructed by three-dimensional models of a 
plurality n umber of head parts having arbitr^ shapes. The head-part data is subjected 
to scaling and positional aligning, based on the size and direction in the measurement 
data of the face part of the customer, and is then combined with the measurement data 
of the customer. 

The combination of the head-part data piece with the measurement data piece 
of the customer may generate a new polygon between open edges in both data pieces, 
or an open edge in e ith e r one data piece may be changed to fit an open edge of the 
other data piece. 

The three-dimensional shape of the head part of the customer thus constructed 
is expressed, for example, as a triangle-polygon model. In the three-dimensional 
model of the head part of the customer thus constructed, hair is removed firom the real 
image of the customer as shown in FIG. 4. 

In the step S3, operation of selecting hair-style data stored in the hair-style 
database 7 by a user (the customer or the like) is received by the controller 8. The 
selected hair-style data is read out_from the hair-style database 7, and is supplied to the 
hair style mapping processing section 4. 

In this hair style selection, icons, thumbnails, or the like indicating hair styles 
are displayed on the display device 9 to invite the user to make a selection. The 
selection may be done by moving a cursor or the like on the screen of the display 
device 9 or by inputting a number or the like designating a hair style. 
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In the step S4, the hair style mapping processing section 4 of the image 
processing device maps the supplied hair-style data on the three-dimensional mode of 
the head part of the customer supplied from the three-dimensional head part 
reconstruction processing section 3, and outputs the result to the rendering part 5. In 
the processing of mapping the hair-style data, hair data corresponding to each one of 
the h airs, which is included in the hair-style data, is mapped on the surface of the 
three-dimensional model of the head part, in accordance with the shape of the three- 
dimensional model of the head part, by the same manner as taken for texture mapping. 
Details of the mapping processing will be described later with reference to FIGS. 5 and 
6. 

In the step S5, the rendering section 5 renders the three-dimensional model of 
the head part of the customer, on which the hair-style data has been mapped, to 
generate hair-style-matched image data as shown in FIG. 4, and outputs it to the 
display device 9. The method of rendering adopts texture mapping processing for a 
polygon model with respect to the face part of the three-dimensional model, and also 
rendering processing for a curvature model with respect to the hair part. The rendering 
processing for a curvature model is disclosed in details in, for example, "Simulation 
based on a motion model of a string-like object and CG" by Kobayashi, Morishima, 
and Harashima, Shingakugihou, PRU90-127, pp. 15-20, 1991. 

The display device 9 displays an image corresponding to the inputted hair- 
style-matched image data. The controller 8 invites a user (a customer or the like) to 
input whether or not the user is satisfied with the hair-style-matched image displayed 
on the display device 9. 

At this time, the user can operate the controller 8 to watch the hair-style- 
matched image while changing the viewpoint position and the illumination position. If 
the viewpoint position and the illumination position are chmiged, the rendering section 
5 executes again the rendering processing on the three-dimensional model of the head 
part and the hair-style data, so as to correspond to the changed condition (the 
viewpoint position and the illumination position) and outputs the result to the display 
device 9. 

In the step S5, if an input indicating that the user is not satisfied is received, the 
processing returns to the step S3, and the later processing is repeated. In the step S5, if 
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it is determined that an input indicating the user's satisfaction is received from the user, 

this processing is terminated. 

Next, an explanation will be made of details of the hair-style mapping 

processing based on the hair-style mapping processing section 4 in the step S4. 
5 Beforehand, a structural example of the hair-style mapping processing section 4 will 

be explained with reference to FIG. 5. 

The hair-style data read out_section 21 of the hair-style mapping processing 

section 4 reads hair-style data selected with use of the controller 8, from the hair-style 

database 7, and outputs it to the mapping position calculation section 22. The mapping 
10 position calculation section 22 calculates the position on the three-dimensional model 

where hair data (corresponding to a hair) forming part of the hair-style data should be 

mapped. A hair data interpolation processing section 23 obtains hair data to be 

mapped, by interpolation fixjm the peripheral hair data around the hair-style data. The 

coordinate conversion section 24 maps the interpolated hair data by coordinate 
15 conversion from the object coordinate system of the hair data to the object coordinate 

system of the three-dimensional model of the head part. 

Next, details of the hair style mapping processing of the hair style mapping 

processing section 4 will be explained with reference to the flowchart shown in FIG. 6. 

The mapping position calculation section 22 of the hair style mapping processing 
20 section 4 has already been supplied with hair-style data from the hafr-style data read 

out_section 21. 

The hair-style data supplied to the mapping position calculation section 22 is 
obtained by two-dimensionally arranging hafr data approximated to an object having a 
three-dimensional shape, as shown in FIG. 7. 

25 In the example of FIG. 7A, every hair is approximated to a special curve such 

as a three-dimensional B-spline cwrve or the like. In the example of FIG. 7B, every 
hair is approximated to a polygon model. The addresses in the two-dimensional array 
correspond to mapping coordinate values supphed to the three-dimensional model of 
the head part on which the hair style is mapped, like the texture mapping processing. 

30 For example, if a mapping coordinate (u, v) = (0.1, 0.1) is given to a vertex of a 
predetermined polygon of the three-dimensional model of the head part, the address on 
the hair-style data array of the hair data which is mapped on the vertex is ^.lumax. 
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O.lVmax). Where, the Umax is the number of data pieces in the u-direction on the hair- 
style data array, and Vmax is the number of data pieces in the v-direction on the hair- 
style data array. 

The hair-style data is defined by arranging the hair data having an arbitrary 
three-dimensional shape into a two-dimensional array. Therefore, any arbitrary hair 
style can be expressed, and it is possible to generate hair-style data of a hair style 
which is difficult to cut. ^Hte-A method of generating hair-style data will be described 
later with reference to FIG. 16. 

At first, in the step SI 1, the mapping position calculation section 22 initializes 
the count value v in the longitudinal direction of the hair-style data array to 0.0. In the 
step SI 2, the count value u in the lateral direction of the hair-style data array is 
initialized to 0.0. 

In the step S13, the mapping position calculation section 22 calculates the 
position on the three-dimensional model of the head part where hair data expressed by 
the address (u, v) of the hair-style data array is mapped. 

Explanation will now b e mad e mor e sp e cifioally. F IG. 8 shows an arbitrary 
triangle polygon on the three-dimensional model of the head part. The vertexes A, B, 
and C of the triangle polygon respectively have attribute vectors Aattr, Battr, and Cattr 
which express attributes of the three-dimensional model of the head part. 

The attribute is constructed by an object coordinate value of the three- 
dimensional model of the head part, local coordinate axes at a vertex, and a mapping 
coordinate value of hair. Where vectors which form two arbitrary edges of the triangle 
polygon ABC are vectors VI and V2, the vectors VI and V2 can be expressed by the 
next expression (1) with use of the attribute vectors of respective vertexes. 

VI = Battr - Aattr 

V2 = Cattr -Aattr •.• (1) 

Considering that the vectors VI and V2 are the coordinate axes of the triangle 
polygon ABC, an arbitrary point Ph on the triangle polygon ABC can be expressed by 
the next expression (2). 

Ph = Aattr + aVl+bV2...(2) 

At this time, as shown in FIG. 8, coefficients a and b are defihed in a manner 
that lengths between an arbitrary point Ph and points at each of which a line parallel to 
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one axis and extended from the arbitrary point crosses the other axis are normalized by 
the lengths of axial vectors, respectively. 

Where the mapping coordinate values of the vertexes A, B, and C are 
respectively (Ua, Va), (lib, Vb), and (uc, Vc), attention is paid to the point Ph expressed by 
5 the expression (2), in relation to the elements of the mapping coordinate values. The 
next expression (3) then exists. 

Uh = Ua + a (Ub + Ua) + b (Uc - Ua) 

Vh = Va + a (Vb - Va) + b (Ve - Va) . . . (3) 

The expression (3) exists with respect to an arbitrary point in the (U,V) space. 
10 If the counter value u = Up and the counter value v = Vp, the condition for 

satisfying that the point Ph on the three-dimensional model of the head part where hair 
data should be mapped exists inside the triangle polygon is expressed by the next 
expression (4). 

a + b<l ... (4) 

1 5 That is, in the arbitrary triangle polygon of the three-dimensional model of the 

head part, if the coefficients a and b obtained by substituting Uh = Up and vj, = Vp in the 
expression (3) satisfy the expression (4), the point Ph exists inside the triangle polygon. 
Therefore, the hair data expressed by (Up, Vp) on the hair-style data array is mapped on 
the triangle polygon. Accordingly, the mapping position of the hair data on the three- 

20 dimensional model of the hair data is calculated by using the expression (2) in the 
mapped triangle polygon. 

Returning to FIG. 6, in the step SI 4, the mapping position calculation section 
22 determines whether or not the triangle polygon of the three-dimensional model of 
the head part to be mapped is a hair generation region. This determination is carried 

25 out, based on a flag (positive or negative) which indicates whether or not hair data 
added to each triangle polygon of the three-dimensional model of the head part should 
be mapped. 

If the mapping flag of the triangle polygon including the mapping position 
obtained by the processing in the step S13 is positive, the triangle polygon is 
30 determined to be a hair generation region, and the processing goes to the step SI 5. 

In the step SI 5, the hair data interpolation processing section 23 interpolates 
hair data, which is expressed as (Up, Vp) and should be mapped, with use of hair data 
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stored in the hair-style data array. The interpolation method may use an average value 
of four close points or may use a summation average value of four close points. 
Alternatively, the closest point may be directly used without interpolation. 

FIG. 9 shows a method of using a summation average value from four close 
5 points. Supposing now that the counter value u = Up and the counter value v = Vp, the 
hair-style data array address of the hair data to be mapped is Pa = (mp, np) = (Up Umax, Vp 
Vmax). The hair data at the point Pa is Hp, and the close 4 points are PI to P4 as shown 
in FIG. 9. The hair data pieces at the point PI to P4 are respectively HI to H4. 

In the summation average method from 4 close points, the hair data Hp to be 
10 mapped is interpolated from the data of the 4 close points with use of the next 
expression (5). 

Hp = dm (dn H2 + (1-dn) HI) + (1-dm) (dn H4 + (1-dn) H3)) ... (5) 
If the hair data is a special curve such as a three-dimensional B-spline curve or 
the like as shown in FIG. 7A, the interpolation processing of the hair data is obtained 

15 by interpolation of data of each control point. Meanwhile, if the hair data is a polygon 
model as shown in FIG. IB, the interpolation processing of the hair data is obtained by 
interpolation between respective polygon vertex data pieces. In the expression (4), dm 
is the decimal part of mp, and dn is the decimal part of Up. 

Returning to FIG. 6, in the step SI 6, the coordinate conversion section 24 

20 coordinate converts hair data subjected to interpolation processing, from the object 
coordinate system of the hair data into the object coordinate system of the three- 
dimensional model of the head part. 

The concept of the coordinate conversion will be explained with reference to 
FIGS. lOA and lOB. Suppose that an arbitrary hair data piece on the hair-style data 

25 array is H. The hair data H is expressed as (I, J, K) according to the object coordinate 
system of the hair data. The object coordinate system (I, J, K) of the hair data may be 
arbitrarily given in case of a three-dimensional coordinate system in which the start 
point of the hair data H is an origin. For example, this is an orthogonal coordinate 
system in which the lateral direction of the hair-style data array is I, the longitudinal 

30 direction thereof is J, and a direction perpendicular to these directions is K. Thus, the 
start point of the hair data H is expressed as (I, J, K) = (0, 0, 0), and the other control 
points are expressed by relative positions from the start point of the hair data. 
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Meanwhile, the three-dimensional model of the head part is expressed as an 
object coordinate system (X, Y, Z). The object coordinate system of the three- 
dimensional model of the head part is generally a three-dimensional orthogonal 
coordinate system, e.g., an orthogonal coordinate system in which the direction toward 
5 the right ear from the gravity center position of the head part as an origin is X, the 
direction toward the top of the head is Y, and the direction toward the nose is Z. 

The coordinate conversion section 24 converts the hair data H expressed by the 
object coordinate system (I, J, K) of the hair data into a local coordinate system (T, S, 
N) at the mapping position Ph of the three-dimensional model of the head part obtained 
10 in the step S13. The local coordinate system (T, S, N) at this point Ph can be arbitrarily 
determined in case of the three-dimensional coordinate system using the point Ph as an 
origin. For example, it is an orthogonal coordinate system in which N is the vector in 
the normal line direction at the point Ph, T is the vector in the tangent direction, which 
is obtained as a vector product of the vector in the radial direction toward the point Ph 
15 from the center axis (Y-axis in FIG. 10) in the longitudinal direction of the three- 
dimensional model of the head part and the vector N in the normal line direction, and S 
is the vector perpendicular to the normal line direction N and the tangent direction T. 

Suppose that the local coordinate system at each vertex of the three- 
dimensional model of the head part is previously given in the head-part database. 
20 Therefore, like the calculation of the mapping position, the local coordinate system (T, 
S, N) at the point Ph is calculated with use of the expression (2) in the triangle polygon 
to be mapped. Supposing that data at the object coordinate system (I, J, K) of an 
arbitrary control point Cj of the hair data H is (i, j, k), the control point Ci is expressed 
by the next expression (6) in the local coordinate system (T, S, N) at the point Ph. 
25 Ci-iT+jS+kN ...(6) 

Where, the unit vectors of the local coordinate system (T, S, N) respectively 
have values of the object coordinate system (X, Y, Z) of the three-dimensional model 
of the head part. 

Therefore, the control point Q expressed by the expression (6) is expressed by 
30 the object coordinate system (X, Y, Z) of the three-dimensional model of the head part. 

Returning to FIG. 6, in the step SI 7, the mapping position calculation section 
22 determines whether or not the counter value u exceeds 1.0. If the counter value u is 
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not determined to exceed 1.0, the processing goes to the step SI 8. In the step SI 8, the 
mapping position calculation section 22 increase the counter value u by an increment 
value du. The increment value du indicates how long pitch hair data stored in the hair- 
style data array is mapped in the lateral direction. 
5 Thereafter, the processing returns to the step SI 3, and the later processing is 

repeated. If it is determined in the step S17 that the counter value u exceeds 1.0, &is 
means that m apping processing in the lateral direction has r eached the end of one line 
of the hair-style data array, so the processing goes to the step SI 9. 

In the step SI 9, the mapping position calculation section 22 determines 

10 whether or not the counter value v exceeds 1.0. If the counter value v is determined as 
not exceeding 1.0, the processing goes to the step S20. In the step S20, the mapping 
position calculation section 22 increases the counter value v by an increment value dv. 
The increment value dv indicates how long pitch hair data stored in the hair-style data 
array is mapped in the longitudinal direction. 

15 Thereafter, the processing returns to the step S12 and the later processing is 

repeated. In the step SI 9, if the counter value v is determined as exceeding 1.0, tlHS 
means that m apping processing is completed with respect to all Unes. The hair 
mapping processing is terminated, and the processing goes to the step S5 shown in 
FIG. 3. 

20 In the step SI 4, if the mapping flag of the triangle polygon including the 

mapping position obtained by the processing in the step SI 3 is negative, the triangle 
polygon is not determined to be a hair generation region, and the processing of the 
steps S15 and S16 is skipped. 

By thus specifying a hair generation region with use of a mapping flag, 

25 mapping can be effected only on an arbitrary region of the three-dimensional model of 
the head part, without mapping all the hair data pieces that construct the hair-style data 
read out_from the hair-style database 7. Therefore, a hair-style such as a so-called 
Machican cut, a star cut, or the hke can be easily generated. 

Also, according to this hair-style mapping processing, each of the hair-style 

30 data pieces stored in the hair-style database 7 is expressed by the object coordinate 
system of hair and can be mapped on a three-dimensional model of a head part having 
any arbitrary shape, without depending on the shape of the human head part. 
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Next, the data structure of the hair-style database 7 will be explained with 
reference to FIG. 1 1 . The hair-style database 7 is constructed by a ptoahtv n umber of 
hair-style data pieces. Each hair-style data piece is of a two-dimensional array as 
described above, and at the top, the number of data pieces Utnax in the lateral direction 
5 and the number of data pieces v^ax in the longitudinal direction. Following them, hair 
data pieces for (umax x Vmax) are described. 

In some cases, group information (described later) is described at the top of the 
hair data piece. Subsequently, a plurality n umber of control point data pieces are 
described, in the case where hair data is approximated to a three-dimensional 
10 parametric curve or the likcr. The the control point data pieces are expressed by the 
hair object coordinate system (I, J, K). 

The hair data may be three-dimensional polygon data. In this case, a plurahty 
number of polygon vertex data pieces are described as hair data. 



15 group infoimation can be used for hair data interpolation processing in the step S15 of 
the hair-style mapping processing (FIG. 6) described above. 

With reference to FIG. 12, explanation will be made of the hair data 
interpolation processing in cas e w here group information is supplied. Where the 
coimter value u = Up and the counter value v = Vp are given, the address on the hair- 

20 style data array of hair data to be mapped is Pa = (mp, np) = (up Umax, Vp Vmax)- 

Suppose that the hair data at the point Pa is Hp, four close points are points PI 
to P4 as shown in FIG. 12, and hair data pieces at the points PI to P4 are respectively 
HI to H4. Further, 1 is described as group information in the hair data piece HI, and 2 
is described as group information in each of the hair data pieces H2 to H4. That is, in 

25 this example, only the hair data piece HI belongs to a group 1, and the other hair data 
pieces H2 to H4 belong to a group 2. If group information is given, only the hair data 
pieces that belong to one same group are used to perform interpolation, in the hair data 
interpolation processing. 



30 at first, the closest point if obtained and the group information thereof is obtained. In 
case of FIG. 12, the closest point is tiie point P3. Next, among the four close points PI 



The group information described in the hair data 




More specifically, among the four close points of the point Pa which is m^ped 
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to P4, the number of close points that belong to the same group as that of the closest 
point is obtained. 

If no other point among the four close points PI to P4 than the close point 
belongs to the same group as that of the closest point, the hair data piece of the closest 
5 point is directly used also for the point Pa- 

If two close points including the closest point among the four close points PI to 
P4 belong to the same group as that of the closest point, a linear interpolation value 
between the two points is used for interpolation. 

If three close points including the closest point among the four close points PI 
10 to P4 belong to the same group as that of the closest point, i.e., in the case shown in 
FIG. 12, an interpolation value is calculated by the next expression (7). 

Hp = (l-r)H32 + rH34 

H32 = dnH2 + (l-dn)H3 

H34 = dn H4 + (1-dn) H3 
15 r = (mp - m32)/(m34 - m32) ... (7) 

In this expression, dn is the decimal part of Up, m32 is the m-address at the 
point P32, and m34 is the m-address at the point P34. 

By thus describing group information in hair data, for example, it is possible to 
express even a hair style having parting of hair where interpolation will cause a 
20 problem. That is, if grouping at the boundary of parting is performed, hair data pieces 
in both sides of the boundary can be used to prevent generation of unnatural hair, in 
case where four close points spread over the boundary. 

Next, with reference to FIGS. 12 and 13, explanation will be made of an 
applicative example of the coordinate conversion in the processing in the step S16 in 
25 the hair-style mapping processing (FIG. 6) described above. 

In the coordinate conversion processing in the step SI 6, unevenness in a hair 
style can be expressed with use of random numbers. FIG. 13 A shows a state where 
unevenness in a hair style is not expressed while FIG. 13B shows an example of a state 
in which unevenness in a hair style is expressed by rotating the local coordinate system 
30 with use of a random number in the coordinate conversion. 

Specifically, the local coordinate system (T, S, N) of the head part is rotated 
about an axis in correspondence with a generated random niraiber where the axis is N. 
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Thereafter, the result is converted into the object coordinate system (X, Y, Z) of the 
head part model. By this processing, the directions in which respective hairs grow can 
be changed delicately, without changing the positions where hair data should be 

mapped. 

5 FIG. 14A shows a state in which unevenness in a hair style is not expressed, 

while FIG. 14B shows an example in which imevenness in a hair style is expressed by 
changing mapping positions by random numbers in the coordinate conversion. 

Specifically, the position of Ph as the origin of the local coordinate system (T, 
S, N) is shifted to Pn in correspondence with a generated random number, and is 
10 thereafter converted into the object coordinate system (X, Y, Z) of the head part 
model. By this processing, the position where the hair grows can be changed 
delicately without changing the hair style. 

The technique of rotating the ads of the coordinate system to be converted use 
the random number and/or moving its origin to express natural unevenness in a 
1 5 synthesized image can be apphed to other cases than the synthesis of hair images. 

Meanwhile, in this hair synthesis system, hair-style data can be mapped after 
changing the three-dimensional model of the head part into an arbitrary shape by the 
three-dimensional head part reconstruction processing section 3. FIG. 15A shows a 
state in which the three-dimensional model of a head part is not changed, while FIG. 
20 15B shows a state in which the three-dimensional model of the head part is magnified 
1.5 times in the Y-axis direction. 

Magnification/minification processing for a three-dimensional model of a head 
part is achieved by affine conversion processing. At this time, similar affine 
conversion processing is performed also on the local coordinate system (T, S, N) 
25 assigned to an arbitrary polygon vertex of the three-dimensional model of a head part. 

As shown in FIG. 15B, in the case of magnifying the three-dimensional model 
of the head part for 1.5 times in the Y-axis direction, vectors of T, S, and N are 
multiplied by a conversion matrix for 1.5 times magnification in the Y-axis direction, 
to obtain a local coordinate system (T', S', N') expanded in the longitudinal direction. 
30 By mapping hair data H on the local coordinate system (T', S', N') thus deformed, the 
hair style is magnified 1.5 times in the Y-axis direction, into a shape matched with the 
head part. By this processing, same hair-style data can be mapped on the three- 
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dimensional model of a head part, which is defonned into an arbitrary shape. Next, 
generation of hair-style data will be explained with reference to FIG. 16. FIG. 16 
shows a columnar coordinate system which surrounds a head part. In the columnar 
coordinate system, the axis penetrating the center of a column is taken as the Y-axis, 
5 the distance from the Y-axis to a plane perpendicular to the Y-axis is r, and the angle 
from a reference direction on this plane is 0. 

An arbitrary point Ph on the surface of the head part can be expressed as Ph = 
(yp, 0p) with use of the columnar coordinate system. Further, if the line defined by 
Y = yp and G = 0p of the colunmar coordinate system always has one contact point with 
10 the surface of the head part, the arbitrary point Pj, on the surface of the head part can be 
expressed by (yp, 9p). 

That is, if the shape of the surface of the head part is always convex toward the 
circumferential direction, the above condition (the hne defined by Y= yp and 0 = Op of 
the coluitmar coordinate system always has one contact point with the surface of the 
1 5 head part) is satisfied. 

Considering the scalp part excluding the face, the shape of the surface of the 
head part is generally considered to be always convex toward the circumferential 
direction. Therefore, the arbitrary point Ph on the scalp part of the head-part model 
can be expressed by Ph = (yp, 0p). 
20 Creation of hair-style data can be relatively easily carried out by using a system 

disclosed in "Development of a hair design system using tuft models" by Kishi, 
Saegusa, and Morishima, in Shingakugihou, MVE97-102, pp. 67-74, 1998. In the 
system, hair-style data is prepared by editing hair data generated on a given head-part 
model. Hair data outputted from this system is expressed by the object coordinate 
25 system of the given head-part model. 

The hair database 7 in the present invention records hair-style data which is 
stored in a two-dimensional array by mapping the hair data on a space expressed by (y, 
0) of a columnar coordinate system. For example, the following expression (8) is used 
for conversion from the (y, 0) space into the (u, v) space of the two-dimensional array 
30 as shown in FIG. 7. 

u = 0p / 27C 

v = (yp-y.rin)/(ymax-ynu„)...(8) 
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In this expression, ymm is the smallest value of the Y-axis of the columnar 
coordinate system of the surface of the head part, and ymax is the largest value of the Y- 
axis of the columnar coordinate system of the surface of the head part. 

The conversion from the (y, 0) space into the (u, v) space may be of non-linear. 
For example, sampling near the top part of the head can be prevented from being 
coarse, as long as the conversion is arrmged such that the increment in the v-direction 
becomes smaller as it comes closer to the head top. 

As described above, hair-style data consisting of hair data converted into the 
(u, v) space is recorded in the hair-style database 7. 

The present invention is also applicable to the case of expressing a string-like 
object other than hair in CG. 

Meanwhile, the T he series of processing in the image processing apparatus 2 as 
described above can be executed by both hardware and software. In case of executing 
the series of processing by software, the software can be installed from a recording 
medium into a computer in which the program constituting the software is 
incorporated in dedicated hardware or a general-purpose computer which can execute 
various fimctions computer by installing various programs. 

FIG. 17 shows a structural example of a personal computer which realiz e s 
embodies t he image processing apparatus 2. This personal computer includes a CPU 
(Central Processing Unit) 41. The CPU 41 is connected with an input/output interface 

45 through a bus 44. The input/output interface 45 is connected with: an input section 

46 constructed by an input device such as a keyboard, a mouse, or the like; an image 
output section 47 for outputting a hair-style-matched image to the display device 9; a 
storage section 48 constructed by a hard-disc drive for storing a program, various data, 
and the hke; a communication section 49 for communicating data through Internet or 
the like; and a drive 50 for reading/writing data with respect to a recording medium 
such as a magnetic disc 51, a semiconductor memory 54, or the like. The bus 44 is 
connected with a ROM (Read Only Memory) 42 and a RAM (Random Access 
Memory) 43. 

An image processing program which lets feis -the p ersonal computer execute 
operation as the image processing apparatus 2 is suppUed in the form that it is stored in 
the magnetic disc 51 or semiconductor memory 54, to the personal computer. It is 
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read by the drive 50 and installed in the hard disc drive included in the storage section 
48. The image processing program installed in the storage section 48 is loaded from 
the storage section 48 to the RAM 43 and executed, in accordance with instructions 
from the CPU 41 corresponding to commands from the user. 

In the present Bp e oifioation i nvention. the step of describing the program 
recorded in the recording medium includes not only processing which is carried out 
based on time in accordance with a described order but also processing which is 
executed in parallel or individually although it is not always performed based on time. 

Also, in the present Gpeoification i nvention, the term of "system" m e ano 
includes the entire apparatus constructed b¥-of a plurality n umber of apparatuses 
and/ or devices. 
Industrial Applicability 

As described above, according to the present invention, the hair-style is stored, 
stored hair-style data is read out, and hair data corresponding to a hair and included in 
the hair-style data is mapped at a predetermined position of a three-dimensional shape 
image of a head part. Therefore, a hair style consisting of hairs every of w hich is-are 
expressed by CG can be changed so as to correspond to phgal -a number of different 
head shapes. Also, according to the present invention, stored hair-style data is read 
out, and hair data corresponding to a hair and included in the read out_hair-style data is 
mapped at a predetermined position on a three-dimensional shape image of a head 
part. Based on the mapping results, a hair-style-matched image is generated. 
Therefore, a hair style consisting of hairs 0ver>'^ of w hich is -are expressed by CG can 
be changed so as to correspond to phaal -a number of different head shapes. 

Although the present invention has been described with reference to specific 
embodiments, those of skill in the art will recognize that changes may be made thereto 
without departing from the spirit and scope of the invention as set forth in the hereafter 
appended claims. 
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ABSTRACT OF THE DISCLOSURE 
The image processing apparatus changes a hair style of hairs e very of w hich is 
are expressed by computer graphics (" CG"), so as to correspond to a pte=aj -number of 
different head part shapes. The apparatus oompriGeo includes : a three-dimensional head 
part reconstruction processing section (3^for synthesizing three-dimensional shape 
data of a face part inputted from an image pickup device fl^with three-dimensional 
shape data of a head part read out from a head-part database-(6), to construct a three- 
dimensional shape model of the neck part; a hair-style mapping processing section (4) 
for mapping hair data read out from a hair-style database f59-on the three-dimensional 
shape model of the neck part; and a rendering section ^§)-for rendering the three- 
dimensional shape model of the neck part on which the hair-style data is mapped, to 
generate hair-style-matched image data. The hair-style database f?)-stores plural a 
number of p ieces of hair-style data. 
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In the Claims; 



Please amend Claims 1, 5-11, 15-21, and 25-30 as follows: 
1 . (Once amended! An image processing apparatus for synthesizing a hair 
image with a three-dimensional shape image of a head part, to generate a hair-style- 
5 matched image, comprising: 

a_storage means p art for storing a hair-style data piece constructed by a 
pte ^pluralitv of hair data pieces arranged on a two-dimensional array; 

aLread out meano p art for reading out the hair-style data piece stored in 
the storage meaBspart; 

10 a.mapping means ; ^rLfor mapping ar-the hair data piece corresponding 

to a hair contained in the hair-style data piece read out by the read out means p art , at a 
predetermined position on the three-dimensional shape image of the head part; and 

a generation HieMis-Eart_for generating the hair-style-matched image, 
based on a mapping result obtained by the mapping ffieaaspart. 

15 5. (Once amended) The image processing apparatus according to claim 1, 

wherein the storage meano p art stores the hair-style data piece constructed by plural 
with a plurality of data pieces arranged on a two-dimensional array corresponding to a 
projected image obtained by projecting a three-dimensional hair style expressed by a 
colximnar coordinate system? on a two-dimensional coordinate system. 

20 6. (Once amended) The image processing apparatus according to claim 1, 

furth e r comprioin gw herein an interpolation meeets -part for int e rpolatin gi nterpolates the 
hair data piece at the predetermined position? with use of a hair data piece close to the 
predetermined position. 

7. (Once amended) The image processing apparatus according to claim 6, 
25 wherein the interpolation m e ans p art interpolates the hair data piece? based on group 

information contained in the hair data piece close to the predetermined position. 

8. (Once amended) The image processing apparatus according to claim 1, 
wherein the mapping means p art includes a_coordinate conversion m e ans p art for 
converting the hair data piece expressed by a first coordinate system into data of a 

30 second coordinate system by which the three-dimensional shape image of the head part 
is expressed. 
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9. (Once amended) The image processing apparatus according to claim 8, 
wherein the coordinate conversion eieaas -part converts the hair data piece expressed 
by the first coordinate system into data of a local coordinate system having an origin 
on a surface of the head part, and thereafter converts the data into data of the second 

5 coordinate system by which the three-dimensional shape image of the head part is 
expressed. 

10. (Once amended) The image processing apparatus according to claim 8, 
wherein when the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 

1 0 image of the head part is expressed, the coordinate conversion meaas -part executes at 
least one of a.coordinate axis rotation and origin shift, based on a random number. 

11. ( Once amended) An image processing method for synthesizing a hair 
image with a three-dimensional shape image of a head part, to generate a hair-style- 
matched image, the method comprisin g the steps of 

15 a storage step of storing a hair-style data piece constructed by plural a 

plurality of h air data pieces arranged on a two-dimensional arra y is stored ; 

a read out step of r eading out the hair-style data piece stored by 
processing in th e otorage step ; 

a mapping step of m apping a-the hair data piece corresponding to a hair 
20 contained in the read out hair-style data piece road out by prooeooing in th e r e ad out 
step, at a predetermined position on the three-dimensional shape image of the head 
part; and 

a gen e ration step of generating the hair-style-matched image? based on 
ar-the mapping resul t obtained by proooo i jing in the mapping otep . 

25 15. (Once amended) The image processing method according to claim 11, 

wherein in the proc e ssing i n during the storage step of storing , the hair-style data piece 
is constructed by a plural ity of hair data pieces arranged on a two-dimensional array 
corresponding to a projected image obtained by projecting a three-dimensional hair 
style expressed by a columnar coordinate system, on a two-dimensional coordinate 

30 system, and is stored. 

16. (Once amended) The image processmg method according to claim 11, 
further comprising an int e rpolation t he step of interpolating the hair data piece at the 
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predetermined positiony with use of a hair data piece close to the predetermined 
position. 

17. rOnce amended) The image processing method according to claim 16, 
wherein in procopoing i n during the raterpolation step of interpolating, the hair data 

5 piece is interpolated? based on group information contained in the hair data piece close 
to the predetermined position. 

18. (Once amended) The image processing method according to claim 11, 
wherein the procesGing in t he step of mapping step-includes a coordinate conversion 
step of converting the hair data piece expressed by a first coordinate system into data 

10 of a second coordinate system by which the three-dimensional shape image of the head 
part is expressed. 

19. (Once amended) T he image processing method according to claim 18, 
wherein in proceooing in t he coordinate conversion step includes? converting t he hair 
data piece expressed by the first coordinate system Id conv e rt e d i nto data of a local 

15 coordinate system having an origin on a surface of the head part, and thereafter, 
converting t he data is converted into data of the second coordinate system by which 
the three-dimensional shape image of the head part is expressed. 

20. (Once amended) The image processing method according to claim 18, 
wherein wfaea -the step of converting t he hair data piece expressed by the first 

20 coordinate system is converted into the data of the second coordinate system by which 
the three-dimensional shape image of the head part is expressed ? includes executing at 
least one of coordinate axis rotation and origin shi ft io ex e cut e d, based on a random 
number, in processing in the coordinate conversion step. 

21. (Once amended) A recording medium which records a computer- 
25 readable program for image processing of synthesizing a hair image with a three- 
dimensional shape image of a head part, to generate a hair-style-matched image, the 
program comprising: 

a storage st ^part for of storing a hair-style data piece constructed by 
pkorsA -a. plurality of h air data pieces arranged on a two-dimensional array; 
30 a read out step -part for of reading out the hair-style data piece stored by 

processing in the storage stq^part: 
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a mapping et ^part for of mapping a -the h air data piece corresponding 



to a hair contained in the hair-style data piece read out by procesoing in t he read out 
steppart, at a predetermined position on the three-dimensional shape image of the head 
part; and 
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25. (Once amended) The recording medium according to claim 21, 

wherein^ in the during processing in the storage st^part, the hair-style data piece 
constructed by plural w ith a pluraUty of data pieces arranged on a two-dimensional 
r; 10 array corresponding to a projected image obtained by projecting a three-dimensional 
f° hair style expressed by a columnar coordinate system, on a two-dimensional 

M coordinate system^ is stored. 



O 15 data piece at the predetermined position? with use of a hair data piece close to the 
predetermined position. 

^ 27. (Once amended) The recording medium according to claim 26, 

Pj wherein^ jg -during p rocessing in the interpolation st ^part, the hair data piece is 

interpolated? based on group information contained in the hair data piece close to the 
20 predetermined position. 

28. rOnce amended) The recording medium according to claim 21, 
wherein^ ^le-processing in the mapping st ^part includes a coordinate conversion step 
of converting the hair data piece expressed by a first coordinate system into data of a 
second coordinate system by which the three-dimensional shape image of the head p^t 

25 is expressed. 

29. (Once amended) The recording mediimi according to claim 28, 
wherein^ in -during p rocessing in the coordinate conversion ste ppart , the hair data piece 
expressed by the first coordinate system is converted into data of a local coordinate 
system having an origin on a surface of the head part, and thereafter, the data is 

30 converted into data of the second coordinate system by which the three-dimensional 
shape image of the head part is expressed. 
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26. (Once amended) The recording medium according to claim 21 , wherein 
the program further comprises an interpolation step -part for of interpolating the hair 
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30. rOnce amended) The recording medium according to claim 28, 
wherein^ when the hair data piece expressed by the first coordinate system is converted 
into the data of the second coordinate system by which the three-dimensional shape 
image of the head part is expressed, at least one of coordinate axis rotation and origin 
shift is executed, based on a random number, ia -during p rocessing in the coordinate 
conversion st^part. 
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